SelectableChannels and Process API

Roger Riggs Roger.Riggs at
Tue Jul 28 21:33:13 UTC 2015

Hi Peter,

Sorry to have taken so long to get back to this thread on 
SelectableChannels for process I/O.
Working on the pipelines idea got me back to thinking about this proposal.

One bit of spec/implementation to address is the state of the In/Out/Err 
of the process when using Redirect.PIPE_CHANNEL.   It may be as simple as
specifying, as is done for the other Redirect cases, that they have 
'null' implementations and
setting the respective elements of fds[0..2] to -1 before the call to 
initStreams  (line 383//401).

If this moves forward, there should be more tests especially of the 
Selectable functions
and error cases.  Note that 'assert' is not always checked, it depends 
on how the test is invoked.

I've got a busy August, but could spend time on this later (between all 
the other priorities).

Thanks, Roger

On 4/15/2015 2:31 AM, Peter Levart wrote:
>>> So I've been playing with the idea of exposing the "real" pipe 
>>> channels in last couple of days. Here's the prototype I came up with:
>>> This adds new Redirect type to the API and 3 new methods to Process 
>>> that return Pipe channels when this new Redirect type is used. It's 
>>> interesting that no native code changes were necessary. The behavior 
>>> of pipes on Windows is a little different (perhaps because the Pipe 
>>> NIO API uses sockets under the hood on Windows - why is that? 
>>> Windows does have a pipe equivalent). What bothers me is that file 
>>> handles opened on files (when redirecting to/from File) can be 
>>> closed as soon as the subprocess is started and the subprocess is 
>>> still able to read/write from the files (like with UNIX). It's not 
>>> the same with pipe (i.e. socket) handles on Windows. They must be 
>>> closed only after subprocess exits.
>>> If this subtle difference between file handles and socket handles on 
>>> Windows could be dealt with (perhaps some options exist that affect 
>>> subprocess spawning), then the extra waiting thread would not be 
>>> needed on Windows.
>>> So what do you think of this API update?
>> Definitely worthy of a separate thread.  It looks promising and 
>> addresses some of the issues
>> raised, while moving other problems from the implementation to the 
>> application.
>> Such as closing of the channels and cleanup.  I worry about how the 
>> resources are freed
>> if the code spawning the app doesn't do the cleanup.  Will it require 
>> hooks (like a finalizer)
>> to do the cleanup?
>> Also, it doesn't help with Martin's goal of being able to implement
>> emacs in Java since it doesn't provide pty control.
>> As you are aware the complexity in Process is to ensure a timely 
>> cleanup and
>> allowing the Process to terminate and release the process resources
>> when it was done and not having to wait for the stdout/stderr consumer.
> I wonder how this automatic stream cleanup really helps in real-world 
> programs. It doesn't help the Process to terminate and release the 
> process resources any sooner as the process terminates on it's own 
> (unless killed) and OS releases it's resources without the outside 
> help anyway. Draining and closing the stream after the process has 
> already exited just releases one file handle (the consuming side of 
> the pipe) in a promptly manner. This could be left to the user and/or 
> finalizer. Draining after the process has already exited does not help 
> the process to exit any sooner as it happens after the fact. A program 
> that doesn't consume the stream can cause the process to hang forever 
> as the pipe's buffer is bounded (64k typically). So draining and 
> closing after the process has exited only potentially helps for the 
> last 64k of the stream and only to release one file handle in a 
> potentially more timely manner.
> OTOH now that ProcessImpl for UNIX does that (and why does Windows 
> implementation not do that?) sloppy programs might exist that would 
> potentially break if the status quo is not maintained.
> But new functionality need not be so permissive. I'll take a look at 
> how and if Channel(s) do any kind of automatic cleanup based on 
> reachability and whether this can be bolted on for Process use. I 
> doubt it is possible to drain and close a Channel without disturbing 
> the ongoing Selector IO processing...
> Regards, Peter 

More information about the core-libs-dev mailing list