From cowwoc at bbs.darktech.org Sat Aug 1 13:44:35 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Sat, 1 Aug 2009 13:44:35 -0700 (PDT) Subject: AsynchronousSocketChannel.shutdownInput() In-Reply-To: References: <1249087432216-3365782.post@n2.nabble.com> Message-ID: <1249159475968-3370579.post@n2.nabble.com> Alexander Libman wrote: > > I looked at your version of AsynchronousCharByteChannel. > BTW, it is another, example where shutdowOutput() can be used instead of > method write(CharBuffer source, boolean endOfInput) > I think shutdownOutput() provides more consistent interface. > Terabit's AsynchronousCoder uses shutdownOutput() for graceful encoding > final step. > The problem is that write() returns the number of bytes written. shutdownOutput() would only make sense if it returned an Integer (indicate the number of bytes written) instead of returning the channel. Alexander Libman wrote: > > Also there are some cases (not very often) when the closure of external > channel (filter itself) should not close > the embedded(internal) channel. Assume we want to write > AsynchronousSMTPClient. > It can be implemented via the following stack of asynch. filters: > AsynchronousByteChannel > | > AsynchronousCharChannel > | > AsynchronousSMTPChannel > > AsychronousSMTPChannel "rents" the underline channels to do a SMTP > transaction: > MailFrom, RcptTo,..., Body > > After transactions is completed successfully, SMTP channel can be closed, > but > AsynchronousCharChannel can be reused for the next SMTP transaction. > Interesting point. Perhaps there should be a configuration option that indicates whether close() is recursive? Alexander Libman wrote: > > Would you mind to join our effords to facilitate Asynchronous Filter > Framework? > What do you think about splitting functionality of "Asynchronous Char Line > Reader" (i.e. your implementation ) into two filters AsynchronousCoder and > AsynchronousScanner? > The first one will do only read/write CharBuffers and the second can be > generic AsynchronousScanner similar to Scanner? > I like the idea of collaborating on this work and the idea of moving readLine() into AsynchronousScanner, but I'm afraid that the last time I took a look at your framework I was a little put off by the lack of source-code comments and Javadoc. If you take a look at http://projectkenai.com/projects/jperipheral/sources you will notice that I went out of my way to document heavily (in fact, I will probably add more in the future). If we can find a way to work together, great. Otherwise, you are always welcome to "borrow" some of my code as I am using the Apache license. Cheers, Gili -- View this message in context: http://n2.nabble.com/AsynchronousSocketChannel.shutdownInput%28%29-tp3365782p3370579.html Sent from the nio-discuss mailing list archive at Nabble.com. From Alan.Bateman at Sun.COM Sat Aug 1 14:18:47 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 01 Aug 2009 22:18:47 +0100 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <1249077760637-3365317.post@n2.nabble.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> Message-ID: <4A74B137.9020204@sun.com> Gili wrote: > : > I don't understand. Are you talking about adding functionality similar to > Future.get(long timeout) on top of an existing byte channel? I was talking > about the different between: > > read(ByteBuffer dst, A attachment, CompletionHandler > handler); > > and > > void read(ByteBuffer dst, long timeout, TimeUnit unit, A attachment, > CompletionHandler handler); > > As far as I know you can't implement the latter in terms of the former > because you need to pass the timeout values to the underlying OS. What did > you mean? > You should be able to implement the timed variants using the non-timed methods when combined with the timer support in java.util.concurrent. That's what we do in the AsynchronousSocketChannel implementation and should be applicable to other channel types too. All I was suggesting is that this might be something useful to add to your library. -Alan. From Alan.Bateman at Sun.COM Sat Aug 1 14:35:12 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 01 Aug 2009 22:35:12 +0100 Subject: AsynchronousSocketChannel.shutdownInput() In-Reply-To: <1249087432216-3365782.post@n2.nabble.com> References: <1249087432216-3365782.post@n2.nabble.com> Message-ID: <4A74B510.6080002@sun.com> Gili wrote: > Hi, > > Just curious, what's the point of shutdownInput() and shutdownOutput() in > AsynchronousSocketChannel? When would users want to shut them as opposed to > closing the entire channel? > > Thanks, > Gili > The shutdown methods are required to support half-close semantics, important for TCP applications. As Alex said, you can use shutdownOutput to disable further write operations but the channel's socket can continue to receive data from the peer. -Alan. From cowwoc at bbs.darktech.org Sat Aug 1 14:59:19 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Sat, 1 Aug 2009 14:59:19 -0700 (PDT) Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A74B137.9020204@sun.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> Message-ID: <1249163959897-3370772.post@n2.nabble.com> Alan Bateman wrote: > > You should be able to implement the timed variants using the non-timed > methods when combined with the timer support in java.util.concurrent. > That's what we do in the AsynchronousSocketChannel implementation and > should be applicable to other channel types too. All I was suggesting is > that this might be something useful to add to your library. > I wonder if this is a difference between socket communications and serial ports. As far as I know, there is no way for me to initiate a "wait forever" read against a serial port and then fire some method at a later time asking it to return as much data has it managed to gather so far. Looking at http://msdn.microsoft.com/en-us/library/aa363437%28VS.85%29.aspx and http://msdn.microsoft.com/en-us/library/aa363791%28VS.85%29.aspx it seems that if you cancel a read (the only way I know of interrupting it) the operation will return zero bytes. Am I missing something? Thanks, Gili -- View this message in context: http://n2.nabble.com/AsynchronousByteCharChannel-and-Timeouts-tp3363004p3370772.html Sent from the nio-discuss mailing list archive at Nabble.com. From Alan.Bateman at Sun.COM Sun Aug 2 13:07:48 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sun, 02 Aug 2009 21:07:48 +0100 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <1249163959897-3370772.post@n2.nabble.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> Message-ID: <4A75F214.6080105@sun.com> Gili wrote: > : > I wonder if this is a difference between socket communications and serial > ports. As far as I know, there is no way for me to initiate a "wait forever" > read against a serial port and then fire some method at a later time asking > it to return as much data has it managed to gather so far. > > Looking at http://msdn.microsoft.com/en-us/library/aa363437%28VS.85%29.aspx > and http://msdn.microsoft.com/en-us/library/aa363791%28VS.85%29.aspx it > seems that if you cancel a read (the only way I know of interrupting it) the > operation will return zero bytes. Am I missing something? > Cancellation of I/O operations is an awkward topic and clearly not feasible on all operating systems and in all cases. The timeout support in AsynchronousSocketChannel does not require it. This approach might be worth exploring for your serial port channel but it really depends on it if makes sense to want to continue to use the channel after a timeout. In the case of AsynchronousSocketChannel, the timeout support is there to make it easy to deal with cases where the peer does not respond. The typical error handling includes close the channel. In the case of serial I/O, how would you expect the application to handle a timeout? -Alan. From cowwoc at bbs.darktech.org Mon Aug 3 10:23:47 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Mon, 3 Aug 2009 10:23:47 -0700 (PDT) Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A75F214.6080105@sun.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> Message-ID: <1249320227257-3379223.post@n2.nabble.com> Alan Bateman wrote: > > Gili wrote: >> : >> I wonder if this is a difference between socket communications and serial >> ports. As far as I know, there is no way for me to initiate a "wait >> forever" >> read against a serial port and then fire some method at a later time >> asking >> it to return as much data has it managed to gather so far. >> >> Looking at >> http://msdn.microsoft.com/en-us/library/aa363437%28VS.85%29.aspx >> and http://msdn.microsoft.com/en-us/library/aa363791%28VS.85%29.aspx it >> seems that if you cancel a read (the only way I know of interrupting it) >> the >> operation will return zero bytes. Am I missing something? >> > Cancellation of I/O operations is an awkward topic and clearly not > feasible on all operating systems and in all cases. The timeout support > in AsynchronousSocketChannel does not require it. This approach might be > worth exploring for your serial port channel but it really depends on it > if makes sense to want to continue to use the channel after a timeout. > In the case of AsynchronousSocketChannel, the timeout support is there > to make it easy to deal with cases where the peer does not respond. The > typical error handling includes close the channel. In the case of serial > I/O, how would you expect the application to handle a timeout? > Here is a simple counter-example that comes to mind: write() to request a computation on the remote end read() with a timeout waiting for the result if a timeout occurs, write() telling the queue to abort the computation In the above case you could disconnect on timeout, reconnect and request an abort. Forcing a disconnect on a timeout is not the end of the world, but it seems like an unnecessary limitation. Instead of invalidating the channel, couldn't you have specified that in case of a timeout the API will return the number of bytes that were written by the underlying implementation (ideally zero) and leave it up to the user to decide if he wishes to disconnect or do something else? In parallel, I will ask on another mailing list whether they have real-life use-cases of using a serial-port after a timeout. Gili -- View this message in context: http://n2.nabble.com/AsynchronousByteCharChannel-and-Timeouts-tp3363004p3379223.html Sent from the nio-discuss mailing list archive at Nabble.com. From Alan.Bateman at Sun.COM Mon Aug 3 13:14:54 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 03 Aug 2009 21:14:54 +0100 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <1249320227257-3379223.post@n2.nabble.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> Message-ID: <4A77453E.5090103@sun.com> Gili wrote: > : > Here is a simple counter-example that comes to mind: > > write() to request a computation on the remote end > read() with a timeout waiting for the result > if a timeout occurs, write() telling the queue to abort the computation > > In the above case you could disconnect on timeout, reconnect and request an > abort. Forcing a disconnect on a timeout is not the end of the world, but it > seems like an unnecessary limitation. Instead of invalidating the channel, > couldn't you have specified that in case of a timeout the API will return > the number of bytes that were written by the underlying implementation > (ideally zero) and leave it up to the user to decide if he wishes to > disconnect or do something else? > AsynchronousSocketChannel doesn't close the channel on timeout. Rather it just invokes the completion handler's failed method with the InterruptedByTimeoutException. In many cases the application will close the channel at this point because the peer is not responding and there is nothing it can do. The wording in the spec related to continued use of the channel is to deal with possible implementations where it is completely unfeasible to cancel an I/O operation and leave the connection and channel is a consistent state. For example, suppose an implementation is using blocking I/O then there may be thread blocked in read or write and no way to unblock it cleanly. Another example may be where it is possible to cancel an I/O but have no way to guarantee that bytes haven't been transferred in/out of the socket buffer. That would completely break stream communication and clearly this an area where serial port I/O will differ (because you are at lower level and I assume there is some protocol to over errors, right?. In other words, the protocol will recover from data loss and other protocol errors). -Alan. From libman at terabit.com.au Mon Aug 3 21:23:05 2009 From: libman at terabit.com.au (Alexander Libman) Date: Tue, 4 Aug 2009 00:23:05 -0400 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A77453E.5090103@sun.com> Message-ID: In addition to Alan comments. we can consider timed operations as a contract to finish operation in specified time interval. Timeout error can be interpereted as kind of execution error and it leaves the channel in undefined state. This is major difference between asynchronous and sychnoronous non-blocking ways of programming. For the algorithms when the content of the next portion data to send depends depends not only on peer reply, but also on the time of reply arrival or timeouts, we have to inroduce imaginable "AsynchronousTimers" with completion handlers. Currently, we can use Timer and TimerTask in java.util as "AsynchronousTimer" and CompletionHandler. Usually, the Completion Handler invocation works as signal for protocol state machine. Now, we simply add the number of signals to our state machine. The main advantages are : 1) the single TimerTask can be used as signal for several channels, i.e. "watch dog" approach. Would be nice, if the user can facility to register the watch dog time interval and timer task for whole AsynchronousGroup. This task can iterate over group channel and for each channel invoke a special callback. Inside such callback the developer can calculate the time expired from the last started read/write operations. 2) very often we are not interested in time interval for particular I/O operation, but rather we are inetresrted in transition from state A to state B in specified time interval. Between states A and B, it can be executed a lot of read/write operations. AsynchronousTimer approach allows to resolve this task and keep the channel in consistent state if we decide to change our actions in timed manner. I am not sure do we want timed operations for artificial AsynchronousChannels (Filters). Example, AsynchronousSSLChannel. We have different timeout meaning for external and internal channels. Support of general timeouts for the filters -I think it is too much overhead. Independent AsynchronousTimer makes life easy here as well. For SerialPort communications AsynchrnousTimer aprproach works too. You can setup: "A value of zero for both the ReadTotalTimeoutMultiplier and ReadTotalTimeoutConstant members indicates that total time-outs are not used for read operations." Or it is possible to use both Windows Comm timeouts to control the rate of arriving bytes (like for low level data link protocol ) and high level-timeouts "Timers" for the application layer. Alex > -----Original Message----- > From: nio-discuss-bounces at openjdk.java.net > [mailto:nio-discuss-bounces at openjdk.java.net]On Behalf Of Alan Bateman > Sent: Monday, August 03, 2009 4:15 PM > To: Gili > Cc: nio-discuss at openjdk.java.net > Subject: Re: AsynchronousByteCharChannel and Timeouts > > > Gili wrote: > > : > > Here is a simple counter-example that comes to mind: > > > > write() to request a computation on the remote end > > read() with a timeout waiting for the result > > if a timeout occurs, write() telling the queue to abort the computation > > > > In the above case you could disconnect on timeout, reconnect > and request an > > abort. Forcing a disconnect on a timeout is not the end of the > world, but it > > seems like an unnecessary limitation. Instead of invalidating > the channel, > > couldn't you have specified that in case of a timeout the API > will return > > the number of bytes that were written by the underlying implementation > > (ideally zero) and leave it up to the user to decide if he wishes to > > disconnect or do something else? > > > AsynchronousSocketChannel doesn't close the channel on timeout. Rather > it just invokes the completion handler's failed method with the > InterruptedByTimeoutException. In many cases the application will close > the channel at this point because the peer is not responding and there > is nothing it can do. The wording in the spec related to continued use > of the channel is to deal with possible implementations where it is > completely unfeasible to cancel an I/O operation and leave the > connection and channel is a consistent state. For example, suppose an > implementation is using blocking I/O then there may be thread blocked in > read or write and no way to unblock it cleanly. Another example may be > where it is possible to cancel an I/O but have no way to guarantee that > bytes haven't been transferred in/out of the socket buffer. That would > completely break stream communication and clearly this an area where > serial port I/O will differ (because you are at lower level and I assume > there is some protocol to over errors, right?. In other words, the > protocol will recover from data loss and other protocol errors). > > -Alan. > No virus found in this incoming message. > Checked by AVG - www.avg.com > Version: 8.5.392 / Virus Database: 270.13.43/2280 - Release Date: > 08/03/09 17:56:00 > From libman at terabit.com.au Mon Aug 3 21:53:30 2009 From: libman at terabit.com.au (Alexander Libman) Date: Tue, 4 Aug 2009 00:53:30 -0400 Subject: AsynchronousSocketChannel.shutdownInput() In-Reply-To: <1249159475968-3370579.post@n2.nabble.com> Message-ID: > > > > The problem is that write() returns the number of bytes written. > shutdownOutput() would only make sense if it returned an Integer (indicate > the number of bytes written) instead of returning the channel. write() must return the number of elements (chars, not bytes!) processed from the buffer(CharBuffer), i.e. consumed by encoder. How many bytes encoder produces on output(i.e. on internal channel ) is not important for the caller. "write () "is a job and "completed()" callback means the job is done. If write() completed with less bytes processed than requested, it should be called again. write() in TCP means data accepted for the trasnmission, but it does not mean data is delivered. It is up to decoder to buffer encoded data or to push them into internal channel. So shutdownOutput() works like flush and prohibits any future write operations. If error occurs during shutdownOutput() , exception should be thrown. If you are still worried about all data are transmitted on internal channel, you can wait in external channel close() the completion of all operations on internal channel. > > > Alexander Libman wrote: > > > > Also there are some cases (not very often) when the closure > of external > > channel (filter itself) should not close > > the embedded(internal) channel. Assume we want to write > > AsynchronousSMTPClient. > > It can be implemented via the following stack of asynch. filters: > > AsynchronousByteChannel > > | > > AsynchronousCharChannel > > | > > AsynchronousSMTPChannel > > > > AsychronousSMTPChannel "rents" the underline channels to do a SMTP > > transaction: > > MailFrom, RcptTo,..., Body > > > > After transactions is completed successfully, SMTP channel can > be closed, > > but > > AsynchronousCharChannel can be reused for the next SMTP transaction. > > > > Interesting point. Perhaps there should be a configuration option that > indicates whether close() is recursive? May be some flag : close (boolean doNotCloseInternalChannel) and close () by default is implemented as { close (false);} or more generic closeWithOptions(...) to specify: a) close or nor the internal channel b) wait to all operations on internal channel to be completed() > > > > > I like the idea of collaborating on this work and the idea of moving > readLine() into AsynchronousScanner, but I'm afraid that the last time I > took a look at your framework I was a little put off by the lack of > source-code comments and Javadoc. Yes, I know - it is still a draft, but it is working draft and I hope the code is not hard to read. We will definetely try to find time to write better comments/doc. Alex From cowwoc at bbs.darktech.org Tue Aug 4 20:06:35 2009 From: cowwoc at bbs.darktech.org (cowwoc) Date: Tue, 04 Aug 2009 23:06:35 -0400 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: References: Message-ID: <4A78F73B.8060900@bbs.darktech.org> Alexander Libman wrote: > In addition to Alan comments. > we can consider timed operations as a contract to finish operation in > specified time interval. > Timeout error can be interpereted as kind of execution error and it leaves > the channel in undefined state. I don't think that it is safe to assume that timeouts are a form of execution error. When I asked about timeouts on the RXTX mailing list a lot of people responded that they expect to be able to use the channel after a read() timeout. For example, some comport protocols "frame" data packets using a timeout. So you read() and if a timeout occurs you assume that the *next* read denotes a new packet or connection. In such a case, you do not want to close and reopen the comport because you might miss some bytes. I think it is perfectly reasonable for some channel implementations to invalidate after a timeout, if this is the exception to the rule. What worries me is that you seem to be implying the opposite: that more-often-than-not a timeout error should result in the channel in an undefined state. For some category of channels, such as comports, an implementation that invalidates after a timeout is fairly useless. > For the algorithms when the content of the next portion data to send > depends depends not only on peer reply, > but also on the time of reply arrival or timeouts, we have to inroduce > imaginable "AsynchronousTimers" with completion handlers. > Currently, we can use Timer and TimerTask in java.util as > "AsynchronousTimer" and CompletionHandler. > > Usually, the Completion Handler invocation works as signal for protocol > state machine. > Now, we simply add the number of signals to our state machine. > The main advantages are : > 1) the single TimerTask can be used as signal for several channels, i.e. > "watch dog" approach. > Would be nice, if the user can facility to register the watch dog time > interval and timer task for whole AsynchronousGroup. > This task can iterate over group channel and for each channel invoke a > special callback. > Inside such callback the developer can calculate the time expired from the > last started read/write operations. I'm not sure I understand you correctly, but if I am then this isn't ideal from a usability point of view. I want to read 1-n bytes within X milliseconds or timeout with 0 bytes read. This sort of thing should be implemented under the hood (since it's a common use-case) instead of forcing me to implement it on top of the API in terms of timers. > 2) very often we are not interested in time interval for particular I/O > operation, but rather we are inetresrted in transition from state A to state > B in specified time interval. > Between states A and B, it can be executed a lot of read/write operations. > AsynchronousTimer approach allows to resolve this task and keep the channel > in consistent state if we decide to change our actions in timed manner. > > I am not sure do we want timed operations for artificial > AsynchronousChannels (Filters). Example, AsynchronousSSLChannel. > We have different timeout meaning for external and internal channels. > Support of general timeouts for the filters -I think it is too much > overhead. > Independent AsynchronousTimer makes life easy here as well. > > For SerialPort communications AsynchrnousTimer aprproach works too. > You can setup: "A value of zero for both the ReadTotalTimeoutMultiplier and > ReadTotalTimeoutConstant members indicates that total time-outs are not used > for read operations." > Or it is possible to use both Windows Comm timeouts to control the rate of > arriving bytes (like for low level data link protocol ) and high > level-timeouts "Timers" for the application layer. I'm sorry. I couldn't understand what you're trying to say here :( Gili From cowwoc at bbs.darktech.org Tue Aug 4 20:27:28 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Tue, 4 Aug 2009 20:27:28 -0700 (PDT) Subject: AsynchronousSocketChannel.shutdownInput() In-Reply-To: References: <1249087432216-3365782.post@n2.nabble.com> <1249159475968-3370579.post@n2.nabble.com> Message-ID: <1249442848818-3388926.post@n2.nabble.com> Alexander Libman wrote: > > write() must return the number of elements (chars, not bytes!) processed > from the buffer(CharBuffer), i.e. consumed by encoder. > How many bytes encoder produces on output(i.e. on internal channel ) is > not > important for the caller. > "write () "is a job and "completed()" callback means the job is done. > If write() completed with less bytes processed than requested, it should > be > called again. > write() in TCP means data accepted for the trasnmission, but it does not > mean data is delivered. > It is up to decoder to buffer encoded data or to push them into internal > channel. > So shutdownOutput() works like flush and prohibits any future write > operations. > If error occurs during shutdownOutput() , exception should be thrown. > If you are still worried about all data are transmitted on internal > channel, > you can wait in external channel close() the completion of all operations > on internal channel. > Hi Alex, To be honest I don't fully understand the implications of CharsetEncoder.encode(endOfInput=true) and CharsetEncoder.flush(). If I write() 5 characters with endOfInput = false, 1) What's the point of CharsetEncoder.flush() if write(endOfInput=true) already knows that you're flushing? Is it because flush() is overridable by subclasses whereas encode() is not? 2) Am I guaranteed that the total characters outputted by CharsetEncoder.write() and flush() will be equal to 5 characters? I ask because the specification for flush() makes it seem as if there is no limit to the number of bytes it could output. 3) If I want to implement a method similar to OutputStream.flush() on top of CharsetEncoder, how would I do it? write(endOfInput=true) and flush() cannot be invoked more than once, whereas OutputStream.flush() can. But if I don't invoke them am I really flushing? Alexander Libman wrote: > > Yes, I know - it is still a draft, but it is working draft and I hope > the > code is not hard to read. > We will definetely try to find time to write better comments/doc. > Okay. I will try to finalize my implementation in the coming weeks. Once that's done we'll discuss folding it into nio2 or your framework. I'd like to hand maintenance over to someone else once I finish the stabilizing the code. Gili -- View this message in context: http://n2.nabble.com/AsynchronousSocketChannel.shutdownInput%28%29-tp3365782p3388926.html Sent from the nio-discuss mailing list archive at Nabble.com. From Alan.Bateman at Sun.COM Wed Aug 5 01:15:01 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 05 Aug 2009 09:15:01 +0100 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A78F73B.8060900@bbs.darktech.org> References: <4A78F73B.8060900@bbs.darktech.org> Message-ID: <4A793F85.3060907@sun.com> cowwoc wrote: > : > I don't think that it is safe to assume that timeouts are a form > of execution error. When I asked about timeouts on the RXTX mailing > list a lot of people responded that they expect to be able to use the > channel after a read() timeout. For example, some comport protocols > "frame" data packets using a timeout. So you read() and if a timeout > occurs you assume that the *next* read denotes a new packet or > connection. In such a case, you do not want to close and reopen the > comport because you might miss some bytes. > > I think it is perfectly reasonable for some channel > implementations to invalidate after a timeout, if this is the > exception to the rule. What worries me is that you seem to be implying > the opposite: that more-often-than-not a timeout error should result > in the channel in an undefined state. > > For some category of channels, such as comports, an implementation > that invalidates after a timeout is fairly useless. AsynchronousSocketChannel is intended to represent a connection to a reliable stream. Applications using the channel are not expected to tolerate data loss or data repetition in the stream. That paragraph on timeouts is just saying that if, on timeout, that the implementation cannot guarantee that bytes haven't been (or won't be) transferred then it should prevent further I/O of that type on the channel. Also remember the channel may have very different implementations, depending on the operating system and I/O facility used. If implemented using blocking or asynchronous I/O (for example) then it may be completely unfeasible to cancel the I/O operation cleanly when the timer has expired. On the other hand, there will be implementations (non-blocking based in particular) where a timer can safely "cancel" with the guarantee that no bytes have been transferred. In those cases, the channel can of course be re-used. I can't say if this approach is completely suitable for your serial port channel but it does seem like the timeout (or the Future cancel if that style is used) will need to be able to cause the underlying I/O to be cancelled. -Alan. From cowwoc at bbs.darktech.org Wed Aug 5 07:17:43 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Wed, 5 Aug 2009 07:17:43 -0700 (PDT) Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A793F85.3060907@sun.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> <4A77453E.5090103@sun.com> <4A78F73B.8060900@bbs.darktech.org> <4A793F85.3060907@sun.com> Message-ID: <1249481863674-3392148.post@n2.nabble.com> Alan Bateman wrote: > > AsynchronousSocketChannel is intended to represent a connection to a > reliable stream. Applications using the channel are not expected to > tolerate data loss or data repetition in the stream. That paragraph on > timeouts is just saying that if, on timeout, that the implementation > cannot guarantee that bytes haven't been (or won't be) transferred then > it should prevent further I/O of that type on the channel. Also remember > the channel may have very different implementations, depending on the > operating system and I/O facility used. If implemented using blocking or > asynchronous I/O (for example) then it may be completely unfeasible to > cancel the I/O operation cleanly when the timer has expired. On the > other hand, there will be implementations (non-blocking based in > particular) where a timer can safely "cancel" with the guarantee that no > bytes have been transferred. In those cases, the channel can of course > be re-used. I can't say if this approach is completely suitable for your > serial port channel but it does seem like the timeout (or the Future > cancel if that style is used) will need to be able to cause the > underlying I/O to be cancelled. > Okay. I agree with everything you wrote except for the earlier post about how it should be possible to implement read-with-timeout in terms of read-forever. For the win32 comport API at least this doesn't seem to be possible. How does it work for sockets? If you start a read for 10 bytes and a Timer tells you about a timeout after you only read 5 bytes are you saying you can tell the underlying OS to stop reading but also give you whatever data it already read? As I mentioned before, the OS lets me stop ongoing reads for serial ports but then it will queue any read data and I can only get at it if I try to read() again. Gili -- View this message in context: http://n2.nabble.com/AsynchronousByteCharChannel-and-Timeouts-tp3363004p3392148.html Sent from the nio-discuss mailing list archive at Nabble.com. From libman at terabit.com.au Wed Aug 5 22:40:19 2009 From: libman at terabit.com.au (Alexander Libman) Date: Thu, 6 Aug 2009 01:40:19 -0400 Subject: AsynchronousSocketChannel.shutdownInput() In-Reply-To: <1249442848818-3388926.post@n2.nabble.com> Message-ID: Hi Gili, Alan I have read again JavaDoc about encoder and looked at the source code of ChasretEncoder. Basically, its state machine can be decribed as follows: State Signal New State --------- ------------------------- -------------- RESET encode(EOF=false) CODING encode(EOF=true) END reset RESET flush Exception Illegal CODING encode(EOF=false) CODING encode(EOF=true) END reset RESET flush Exception Illegal END encode(EOF=false) Exception Illegal encode(EOF=true) END reset RESET flush FLUSHED FLUSHED encode(EOF=false) Exception Illegal encode(EOF=true) Exception Illegal reset RESET flush FLUSHED This means we should have two kind of filters for AsynchronousCoders : 1) AsynchronousMessageEncoder: 2) AsynchonouseStreamEncoder For the 1) AsynchronousMessageEncoder: each asynchnous write (CharBuffer buffer) translates to the following sequence of operations on encoder: reset(); encode(EOF=true); flush(); shutdownOutput() simply prevents any future writes close() calls shutdownOutput() For the 2) AsynchronousStreamEncoder (Terabit version) reset() called from constructor each write() translates into encode(EOF=false) shutdowOutput() translates into encode(zeroByteBuffer, EOF=true); flush(); and prevents any future writes close() calls shutdownOutput() The type of AsynchronousEncoder (Message or Stream) can be specified at construction time. Another possible option to introduce "HybridEncoder" with additional method flush() that does not have analogy in AsynchronousByteChannel: 3) HybridEncoder works like: reset() called from constructor each write() translates into encode(EOF=false) flush() translates into encode(zeroByteBuffer, EOF=true); flush(); reset() shutdowOutput() translates into encode(zeroByteBuffer, EOF=true); flush(); and prevents any future writes close() calls shutdownOutput() another way to implement HybridEncoder (as Gili proposed): reset() called from constructor each write() translates into encode(EOF=false) each write(EOF=true) translates into encode( EOF=true); flush(); reset() shutdowOutput() translates into encode(zeroByteBuffer, EOF=true); flush(); and prevents any future writes close() calls shutdownOutput() Gili, you missed reset() in your implementation. write(EOF=true) does not mean we can not write anymore, it means end-of-transaction encoding and after flushing is done the encoder is ready for next operations. Any opinions? what interface is preferable? I am thinking that Gili's write(buffer) and write(buffer,EOFFlag) make sense. One more option: setMode(mode) method messageMode -means write() works as 1) and and streamMode - means write() works as 2) setMode () can be invoked at any time. Alex > -----Original Message----- > From: nio-discuss-bounces at openjdk.java.net > [mailto:nio-discuss-bounces at openjdk.java.net]On Behalf Of Gili > Sent: Tuesday, August 04, 2009 11:27 PM > To: nio-discuss at openjdk.java.net > Subject: RE: AsynchronousSocketChannel.shutdownInput() > > > > > Alexander Libman wrote: > > > > write() must return the number of elements (chars, not bytes!) > processed > > from the buffer(CharBuffer), i.e. consumed by encoder. > > How many bytes encoder produces on output(i.e. on internal channel ) is > > not > > important for the caller. > > "write () "is a job and "completed()" callback means the job is done. > > If write() completed with less bytes processed than requested, it should > > be > > called again. > > write() in TCP means data accepted for the trasnmission, but it does not > > mean data is delivered. > > It is up to decoder to buffer encoded data or to push them into internal > > channel. > > So shutdownOutput() works like flush and prohibits any future write > > operations. > > If error occurs during shutdownOutput() , exception should be thrown. > > If you are still worried about all data are transmitted on internal > > channel, > > you can wait in external channel close() the completion of all > operations > > on internal channel. > > > > Hi Alex, > > To be honest I don't fully understand the implications of > CharsetEncoder.encode(endOfInput=true) and CharsetEncoder.flush(). If I > write() 5 characters with endOfInput = false, > > 1) What's the point of CharsetEncoder.flush() if write(endOfInput=true) > already knows that you're flushing? Is it because flush() is > overridable by > subclasses whereas encode() is not? > 2) Am I guaranteed that the total characters outputted by > CharsetEncoder.write() and flush() will be equal to 5 characters? I ask > because the specification for flush() makes it seem as if there > is no limit > to the number of bytes it could output. > 3) If I want to implement a method similar to > OutputStream.flush() on top of > CharsetEncoder, how would I do it? write(endOfInput=true) and > flush() cannot > be invoked more than once, whereas OutputStream.flush() can. But > if I don't > invoke them am I really flushing? > > > Alexander Libman wrote: > > > > Yes, I know - it is still a draft, but it is working draft and I hope > > the > > code is not hard to read. > > We will definetely try to find time to write better comments/doc. > > > > Okay. I will try to finalize my implementation in the coming weeks. Once > that's done we'll discuss folding it into nio2 or your framework. I'd like > to hand maintenance over to someone else once I finish the stabilizing the > code. > > Gili > -- > View this message in context: > http://n2.nabble.com/AsynchronousSocketChannel.shutdownInput%28%29 -tp3365782p3388926.html Sent from the nio-discuss mailing list archive at Nabble.com. No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.392 / Virus Database: 270.13.44/2282 - Release Date: 08/04/09 18:01:00 From Alan.Bateman at Sun.COM Thu Aug 6 02:07:27 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 06 Aug 2009 10:07:27 +0100 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <1249481863674-3392148.post@n2.nabble.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> <4A77453E.5090103@sun.com> <4A78F73B.8060900@bbs.darktech.org> <4A793F85.3060907@sun.com> <1249481863674-3392148.post@n2.nabble.com> Message-ID: <4A7A9D4F.8000408@sun.com> Gili wrote: > : > Okay. I agree with everything you wrote except for the earlier post about > how it should be possible to implement read-with-timeout in terms of > read-forever. Right, this is where we differ because AsynchronousSocketChannel does not require the implementation to have a mechanism to cancel the underlying I/O operation. > For the win32 comport API at least this doesn't seem to be > possible. How does it work for sockets? > It's done using overlapped I/O with WSARecv/WSASend. A completion port is used to receive the notifications when the I/O operations complete. We don't make use of I/O cancellation (Windows Vista did bring a new Win32 call to cancel specific I/O operations but it doesn't provide the guarantees that we require and we don't use it). If it were implemented as blocking sockets then the same issue would arise (from the Win32 docs: "If a send or receive operation times out on a socket, the socket state is indeterminate, and should not be used; TCP sockets in this state have a potential for data loss, since the operation could be canceled at the same moment the operation was to be completed.") > If you start a read for 10 bytes and a Timer tells you about a timeout after > you only read 5 bytes are you saying you can tell the underlying OS to stop > reading but also give you whatever data it already read? As I mentioned > before, the OS lets me stop ongoing reads for serial ports but then it will > queue any read data and I can only get at it if I try to read() again. > A read operation is not required to fill the buffer so if you initiate a read up to 10 bytes then it is normal have the read to complete having read less than 10 bytes. For the case that the timer expires at just around the time that 5 bytes have been read into your buffer then the best solution is to have the I/O operation complete (successfully) and the buffer position moved on by 5 bytes. If you are saying that this is not feasible (because the underlying read returns 0) then are those 5 bytes lost? If this were a stream/socket then this data loss would be a disaster if the application were to read bytes after the timeout and data loss. For a serial connection (we are talking RS232 or descendants, right?) then it's not too bad as there will likely be a high level protocol to detect errors. -Alan. From ullenboom at googlemail.com Thu Aug 6 05:38:14 2009 From: ullenboom at googlemail.com (Christian Ullenboom) Date: Thu, 6 Aug 2009 14:38:14 +0200 Subject: DELETE_ON_CLOSE question In-Reply-To: <4A7308C8.5040103@sun.com> References: <426c3b1a0907310735p12f11dc5w91122c2ba25f82f7@mail.gmail.com> <4A7308C8.5040103@sun.com> Message-ID: <426c3b1a0908060538j3d0098eas9592bc0ba83e51@mail.gmail.com> If I change the Path to Paths.get( "c:/opa.herbert.tmp" ); then its fine. So the combination of Windows + Shared Folders + Samba seems to be the problem. Even if I put some sleep it doesn't delete the file. path.newOutputStream( StandardOpenOption.DELETE_ON_CLOSE, StandardOpenOption.SYNC ).close(); TimeUnit.SECONDS.sleep( 10 ); System.out.println( path.exists() ); // still true! So the file is still there but not if the Path is local, then its correctly deleted. Christian From cowwoc at bbs.darktech.org Thu Aug 6 06:16:35 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 6 Aug 2009 08:16:35 -0500 (CDT) Subject: AsynchronousSocketChannel.shutdownInput() In-Reply-To: References: <1249087432216-3365782.post@n2.nabble.com> <1249159475968-3370579.post@n2.nabble.com> <1249442848818-3388926.post@n2.nabble.com> Message-ID: <1249564595401-3398196.post@n2.nabble.com> Alex, Please take a look at http://kenai.com/projects/jperipheral/sources/svn/content/trunk/java/source/jperipheral/AsynchronousByteCharChannel.java?raw=true I fixed numerous bugs since I posted the source-code on this mailing list. You bring up a good point regarding reset(). Perhaps the class should be refactored as follows: write() invokes endOfInput=false always flush() invokes encoder.write(endOfInput=true), flush(), reset() close() invokes flush() before closing the channel Anyone wishing to send messages versus streams would either use flush() at the end of a message or not. I believe this would result in a cleaner more consistent API with the respect to the core Java APIs. What do you think? Gili Alexander Libman wrote: > > > Hi Gili, Alan > > I have read again JavaDoc about encoder and looked at the source code of > ChasretEncoder. Basically, its state machine can be decribed as follows: > > State Signal New State > --------- ------------------------- -------------- > RESET encode(EOF=false) CODING > encode(EOF=true) END > reset RESET > flush Exception Illegal > > CODING encode(EOF=false) CODING > encode(EOF=true) END > reset RESET > flush Exception Illegal > > END encode(EOF=false) Exception Illegal > encode(EOF=true) END > reset RESET > flush FLUSHED > > FLUSHED encode(EOF=false) Exception Illegal > encode(EOF=true) Exception Illegal > reset RESET > flush FLUSHED > > > This means we should have two kind of filters for AsynchronousCoders : > > 1) AsynchronousMessageEncoder: > 2) AsynchonouseStreamEncoder > > For the 1) AsynchronousMessageEncoder: > each asynchnous write (CharBuffer buffer) translates to the following > sequence of operations on encoder: > reset(); encode(EOF=true); flush(); > shutdownOutput() simply prevents any future writes > close() calls shutdownOutput() > > For the 2) AsynchronousStreamEncoder (Terabit version) > reset() called from constructor > each write() translates into encode(EOF=false) > shutdowOutput() translates into encode(zeroByteBuffer, EOF=true); flush(); > and prevents any future writes > close() calls shutdownOutput() > > The type of AsynchronousEncoder (Message or Stream) can be specified at > construction time. > Another possible option to introduce "HybridEncoder" with additional > method > flush() that does not have analogy in AsynchronousByteChannel: > > 3) HybridEncoder works like: > reset() called from constructor > each write() translates into encode(EOF=false) > flush() translates into encode(zeroByteBuffer, EOF=true); flush(); > reset() > shutdowOutput() translates into encode(zeroByteBuffer, EOF=true); flush(); > and prevents any future writes > close() calls shutdownOutput() > > another way to implement HybridEncoder (as Gili proposed): > reset() called from constructor > each write() translates into encode(EOF=false) > each write(EOF=true) translates into encode( EOF=true); flush(); reset() > shutdowOutput() translates into encode(zeroByteBuffer, EOF=true); flush(); > and prevents any future writes > close() calls shutdownOutput() > > Gili, you missed reset() in your implementation. write(EOF=true) does not > mean we can not write anymore, > it means end-of-transaction encoding and after flushing is done the > encoder > is ready for next operations. > > Any opinions? what interface is preferable? > I am thinking that Gili's write(buffer) and write(buffer,EOFFlag) make > sense. > One more option: > setMode(mode) method > messageMode -means write() works as 1) and > and > streamMode - means write() works as 2) > setMode () can be invoked at any time. > > Alex > > > > > > > > > > > > > > > > > >> -----Original Message----- >> From: nio-discuss-bounces at openjdk.java.net >> [mailto:nio-discuss-bounces at openjdk.java.net]On Behalf Of Gili >> Sent: Tuesday, August 04, 2009 11:27 PM >> To: nio-discuss at openjdk.java.net >> Subject: RE: AsynchronousSocketChannel.shutdownInput() >> >> >> >> >> Alexander Libman wrote: >> > >> > write() must return the number of elements (chars, not bytes!) >> processed >> > from the buffer(CharBuffer), i.e. consumed by encoder. >> > How many bytes encoder produces on output(i.e. on internal channel ) is >> > not >> > important for the caller. >> > "write () "is a job and "completed()" callback means the job is done. >> > If write() completed with less bytes processed than requested, it >> should >> > be >> > called again. >> > write() in TCP means data accepted for the trasnmission, but it does >> not >> > mean data is delivered. >> > It is up to decoder to buffer encoded data or to push them into >> internal >> > channel. >> > So shutdownOutput() works like flush and prohibits any future write >> > operations. >> > If error occurs during shutdownOutput() , exception should be thrown. >> > If you are still worried about all data are transmitted on internal >> > channel, >> > you can wait in external channel close() the completion of all >> operations >> > on internal channel. >> > >> >> Hi Alex, >> >> To be honest I don't fully understand the implications of >> CharsetEncoder.encode(endOfInput=true) and CharsetEncoder.flush(). If I >> write() 5 characters with endOfInput = false, >> >> 1) What's the point of CharsetEncoder.flush() if write(endOfInput=true) >> already knows that you're flushing? Is it because flush() is >> overridable by >> subclasses whereas encode() is not? >> 2) Am I guaranteed that the total characters outputted by >> CharsetEncoder.write() and flush() will be equal to 5 characters? I ask >> because the specification for flush() makes it seem as if there >> is no limit >> to the number of bytes it could output. >> 3) If I want to implement a method similar to >> OutputStream.flush() on top of >> CharsetEncoder, how would I do it? write(endOfInput=true) and >> flush() cannot >> be invoked more than once, whereas OutputStream.flush() can. But >> if I don't >> invoke them am I really flushing? >> >> >> Alexander Libman wrote: >> > >> > Yes, I know - it is still a draft, but it is working draft and I hope >> > the >> > code is not hard to read. >> > We will definetely try to find time to write better comments/doc. >> > >> >> Okay. I will try to finalize my implementation in the coming weeks. Once >> that's done we'll discuss folding it into nio2 or your framework. I'd >> like >> to hand maintenance over to someone else once I finish the stabilizing >> the >> code. >> >> Gili >> -- >> View this message in context: >> http://n2.nabble.com/AsynchronousSocketChannel.shutdownInput%28%29 > -tp3365782p3388926.html > Sent from the nio-discuss mailing list archive at Nabble.com. > No virus found in this incoming message. > Checked by AVG - www.avg.com > Version: 8.5.392 / Virus Database: 270.13.44/2282 - Release Date: 08/04/09 > 18:01:00 > > > -- View this message in context: http://n2.nabble.com/AsynchronousSocketChannel.shutdownInput%28%29-tp3365782p3398196.html Sent from the nio-discuss mailing list archive at Nabble.com. From cowwoc at bbs.darktech.org Thu Aug 6 06:45:31 2009 From: cowwoc at bbs.darktech.org (cowwoc) Date: Thu, 06 Aug 2009 09:45:31 -0400 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A7A9D4F.8000408@sun.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> <4A77453E.5090103@sun.com> <4A78F73B.8060900@bbs.darktech.org> <4A793F85.3060907@sun.com> <1249481863674-3392148.post@n2.nabble.com> <4A7A9D4F.8000408@sun.com> Message-ID: <4A7ADE7B.3020000@bbs.darktech.org> Alan Bateman wrote: > It's done using overlapped I/O with WSARecv/WSASend. A completion port > is used to receive the notifications when the I/O operations complete. > We don't make use of I/O cancellation (Windows Vista did bring a new > Win32 call to cancel specific I/O operations but it doesn't provide the > guarantees that we require and we don't use it). > > If it were implemented as blocking sockets then the same issue would > arise (from the Win32 docs: "If a send or receive operation times out on > a socket, the socket state is indeterminate, and should not be used; TCP > sockets in this state have a potential for data loss, since the > operation could be canceled at the same moment the operation was to be > completed.") Weird. Comports (and sockets too I believe!) can be treated as HANDLEs in the win32 API. You can then use ReadFile(), CancelIo() to read and cancel overlapped operations. CancelIo() has the limitation of only being cancellable from the thread that initiated the read but you can use "completion ports" to work around this limitation and essentially get the Windows Vista functionality back in Windows XP. That's what I'm using for comports. Couldn't you do the same for sockets? > For the case that the timer expires at just around the time that 5 bytes > have been read into your buffer then the best solution is to have the > I/O operation complete (successfully) and the buffer position moved on > by 5 bytes. If you are saying that this is not feasible (because the > underlying read returns 0) then are those 5 bytes lost? To be honest, it's not clear. CancelIo() claims to cancel any pending I/O operations. My interpretation is that if I issue an overlapped read for 10 bytes but CancelIo() after 5 the read bytes would get buffered for future reads (i.e. I don't lose any data). I posted this question to double check: http://stackoverflow.com/questions/1238905/does-cancelio-cause-data-loss > If this were a stream/socket then this data loss would be a disaster if the application > were to read bytes after the timeout and data loss. For a serial > connection (we are talking RS232 or descendants, right?) then it's not > too bad as there will likely be a high level protocol to detect errors. While this is true on a higher level, I can't assume that all serial protocols will have error detection/correction and users would probably refuse to use a library that silently eats bytes ;) At the very least I would need to make such failures more explicit. Gili From Alan.Bateman at Sun.COM Thu Aug 6 08:11:49 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 06 Aug 2009 16:11:49 +0100 Subject: DELETE_ON_CLOSE question In-Reply-To: <426c3b1a0908060538j3d0098eas9592bc0ba83e51@mail.gmail.com> References: <426c3b1a0907310735p12f11dc5w91122c2ba25f82f7@mail.gmail.com> <4A7308C8.5040103@sun.com> <426c3b1a0908060538j3d0098eas9592bc0ba83e51@mail.gmail.com> Message-ID: <4A7AF2B5.7000601@sun.com> Christian Ullenboom wrote: > If I change the Path to Paths.get( "c:/opa.herbert.tmp" ); then its > fine. So the combination of Windows + Shared Folders + Samba seems to > be the problem. Even if I put some sleep it doesn't delete the file. > > path.newOutputStream( StandardOpenOption.DELETE_ON_CLOSE, > StandardOpenOption.SYNC ).close(); > TimeUnit.SECONDS.sleep( 10 ); > System.out.println( path.exists() ); // still true! > > So the file is still there but not if the Path is local, then its > correctly deleted. > > Christian > Good sleuthing! Do you know what version of Samba you are using? I did a quick test with a number of different 2.x and 3.x Samba servers and could duplicate it on one of them (running 3.0.23). Looking through the Samba bugzilla I found 3467 [1] which looks like it might be the issue. I don't have a 3.0 server newer than 3.0.23 to check just now. -Alan. [1] https://bugzilla.samba.org/show_bug.cgi?id=3467 From Alan.Bateman at Sun.COM Fri Aug 7 02:19:50 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 07 Aug 2009 10:19:50 +0100 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A7ADE7B.3020000@bbs.darktech.org> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> <4A77453E.5090103@sun.com> <4A78F73B.8060900@bbs.darktech.org> <4A793F85.3060907@sun.com> <1249481863674-3392148.post@n2.nabble.com> <4A7A9D4F.8000408@sun.com> <4A7ADE7B.3020000@bbs.darktech.org> Message-ID: <4A7BF1B6.3010604@sun.com> cowwoc wrote: > : > Weird. Comports (and sockets too I believe!) can be treated as > HANDLEs in the win32 API. You can then use ReadFile(), CancelIo() to > read and cancel overlapped operations. CancelIo() has the limitation > of only being cancellable from the thread that initiated the read but > you can use "completion ports" to work around this limitation and > essentially get the Windows Vista functionality back in Windows XP. > That's what I'm using for comports. Couldn't you do the same for sockets? No, because it cancels all pending I/O operations on the handle (initiated by the calling thread) and so can't be used when there may be more than one I/O operation outstanding that was initiated from the same thread (meaning read and write). Furthermore, the initiating thread may be executing a completion handler and so may not return to poll the completion port in a timely manner. The CancelIoEx call introduced in Vista does address these limitations but we don't have any plans to use it. The only Vista specific feature we are using here is the new thread agnostic I/O so that I/O operations can be initiated on non-pooled threads. > : > While this is true on a higher level, I can't assume that all > serial protocols will have error detection/correction and users would > probably refuse to use a library that silently eats bytes ;) At the > very least I would need to make such failures more explicit. I'm not suggesting you silently eat bytes, but rather look for a solution that would allow the operation to complete successfully for the case that bytes are transferred into the buffer at just around the time that the timer expires. That would be more desirable than data loss. If you do have data loss, and you do allow the application to read again from the channel then you are dependent on the high-level protocol to recover. Clearly data loss with a stream connection is more fatal. -Alan. From cowwoc at bbs.darktech.org Fri Aug 7 07:58:59 2009 From: cowwoc at bbs.darktech.org (cowwoc) Date: Fri, 07 Aug 2009 10:58:59 -0400 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A7BF1B6.3010604@sun.com> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> <4A77453E.5090103@sun.com> <4A78F73B.8060900@bbs.darktech.org> <4A793F85.3060907@sun.com> <1249481863674-3392148.post@n2.nabble.com> <4A7A9D4F.8000408@sun.com> <4A7ADE7B.3020000@bbs.darktech.org> <4A7BF1B6.3010604@sun.com> Message-ID: <4A7C4133.8020207@bbs.darktech.org> Alan Bateman wrote: > cowwoc wrote: >> : >> Weird. Comports (and sockets too I believe!) can be treated as >> HANDLEs in the win32 API. You can then use ReadFile(), CancelIo() to >> read and cancel overlapped operations. CancelIo() has the limitation >> of only being cancellable from the thread that initiated the read but >> you can use "completion ports" to work around this limitation and >> essentially get the Windows Vista functionality back in Windows XP. >> That's what I'm using for comports. Couldn't you do the same for sockets? > No, because it cancels all pending I/O operations on the handle > (initiated by the calling thread) and so can't be used when there may be > more than one I/O operation outstanding that was initiated from the same > thread (meaning read and write). Furthermore, the initiating thread may > be executing a completion handler and so may not return to poll the > completion port in a timely manner. The CancelIoEx call introduced in > Vista does address these limitations but we don't have any plans to use > it. The only Vista specific feature we are using here is the new thread > agnostic I/O so that I/O operations can be initiated on non-pooled threads. Oops! It seems we've just discovered a bug in my code :) Fixing this is not going to be fun :( I think what I'll have to do is reserve two threads per channel (one for read, one for write) and use a thread pool for handling completion handlers. > I'm not suggesting you silently eat bytes, but rather look for a > solution that would allow the operation to complete successfully for the > case that bytes are transferred into the buffer at just around the time > that the timer expires. That would be more desirable than data loss. Any idea on how to do this? Gili From Alan.Bateman at Sun.COM Fri Aug 7 12:51:24 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 07 Aug 2009 20:51:24 +0100 Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A7C4133.8020207@bbs.darktech.org> References: <1249051228626-3363004.post@n2.nabble.com> <1249051605620-3363044.post@n2.nabble.com> <1249063018213-3364126.post@n2.nabble.com> <4A73424F.3080307@sun.com> <1249077760637-3365317.post@n2.nabble.com> <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> <4A77453E.5090103@sun.com> <4A78F73B.8060900@bbs.darktech.org> <4A793F85.3060907@sun.com> <1249481863674-3392148.post@n2.nabble.com> <4A7A9D4F.8000408@sun.com> <4A7ADE7B.3020000@bbs.darktech.org> <4A7BF1B6.3010604@sun.com> <4A7C4133.8020207@bbs.darktech.org> Message-ID: <4A7C85BC.8060102@sun.com> cowwoc wrote: > : > Any idea on how to do this? How about using WaitForMultipleEvents to multiplex the handle to the serial port and a handle to an event that is set when the timer expires. That should work for the Future usage too as you can set the event when the cancel method is invoked. It would tie up one of the threads in your thread pool for the duration of the I/O operation, but I assume that this wouldn't be a problem as the number of serial ports it likely to be small (I would guess). -Alan. From libman at terabit.com.au Sun Aug 9 20:27:09 2009 From: libman at terabit.com.au (Alexander Libman) Date: Sun, 9 Aug 2009 23:27:09 -0400 Subject: AsynchronousSocketChannel.shutdownInput() In-Reply-To: <1249564595401-3398196.post@n2.nabble.com> Message-ID: Gili, > > > Alex, > > Please take a look at > http://kenai.com/projects/jperipheral/sources/svn/content/trunk/ja > va/source/jperipheral/AsynchronousByteCharChannel.java?raw=true OK, will do > > I fixed numerous bugs since I posted the source-code on this mailing list. > You bring up a good point regarding reset(). Perhaps the class should be > refactored as follows: > > write() invokes endOfInput=false always > flush() invokes encoder.write(endOfInput=true), flush(), reset() > close() invokes flush() before closing the channel yes, later we can add one more method writeAndFlush () Basically its is composition of write and flush, but we can gain in performance a liitle bit > > What do you think? Agree. Alex > > Gili > > > Alexander Libman wrote: > > > > > > Hi Gili, Alan > > > > I have read again JavaDoc about encoder and looked at the source code of > > ChasretEncoder. Basically, its state machine can be decribed as follows: > > > > State Signal New State > > --------- ------------------------- -------------- > > RESET encode(EOF=false) CODING > > encode(EOF=true) END > > reset RESET > > flush Exception Illegal > > > > CODING encode(EOF=false) CODING > > encode(EOF=true) END > > reset RESET > > flush Exception Illegal > > > > END encode(EOF=false) Exception Illegal > > encode(EOF=true) END > > reset RESET > > flush FLUSHED > > > > FLUSHED encode(EOF=false) Exception Illegal > > encode(EOF=true) Exception Illegal > > reset RESET > > flush FLUSHED > > > > > > This means we should have two kind of filters for > AsynchronousCoders : > > > > 1) AsynchronousMessageEncoder: > > 2) AsynchonouseStreamEncoder > > > > For the 1) AsynchronousMessageEncoder: > > each asynchnous write (CharBuffer buffer) translates to the following > > sequence of operations on encoder: > > reset(); encode(EOF=true); flush(); > > shutdownOutput() simply prevents any future writes > > close() calls shutdownOutput() > > > > For the 2) AsynchronousStreamEncoder (Terabit version) > > reset() called from constructor > > each write() translates into encode(EOF=false) > > shutdowOutput() translates into encode(zeroByteBuffer, > EOF=true); flush(); > > and prevents any future writes > > close() calls shutdownOutput() > > > > The type of AsynchronousEncoder (Message or Stream) can be specified at > > construction time. > > Another possible option to introduce "HybridEncoder" with additional > > method > > flush() that does not have analogy in AsynchronousByteChannel: > > > > 3) HybridEncoder works like: > > reset() called from constructor > > each write() translates into encode(EOF=false) > > flush() translates into encode(zeroByteBuffer, EOF=true); flush(); > > reset() > > shutdowOutput() translates into encode(zeroByteBuffer, > EOF=true); flush(); > > and prevents any future writes > > close() calls shutdownOutput() > > > > another way to implement HybridEncoder (as Gili proposed): > > reset() called from constructor > > each write() translates into encode(EOF=false) > > each write(EOF=true) translates into encode( EOF=true); > flush(); reset() > > shutdowOutput() translates into encode(zeroByteBuffer, > EOF=true); flush(); > > and prevents any future writes > > close() calls shutdownOutput() > > > > Gili, you missed reset() in your implementation. > write(EOF=true) does not > > mean we can not write anymore, > > it means end-of-transaction encoding and after flushing is done the > > encoder > > is ready for next operations. > > > > Any opinions? what interface is preferable? > > I am thinking that Gili's write(buffer) and write(buffer,EOFFlag) make > > sense. > > One more option: > > setMode(mode) method > > messageMode -means write() works as 1) and > > and > > streamMode - means write() works as 2) > > setMode () can be invoked at any time. > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> -----Original Message----- > >> From: nio-discuss-bounces at openjdk.java.net > >> [mailto:nio-discuss-bounces at openjdk.java.net]On Behalf Of Gili > >> Sent: Tuesday, August 04, 2009 11:27 PM > >> To: nio-discuss at openjdk.java.net > >> Subject: RE: AsynchronousSocketChannel.shutdownInput() > >> > >> > >> > >> > >> Alexander Libman wrote: > >> > > >> > write() must return the number of elements (chars, not bytes!) > >> processed > >> > from the buffer(CharBuffer), i.e. consumed by encoder. > >> > How many bytes encoder produces on output(i.e. on internal > channel ) is > >> > not > >> > important for the caller. > >> > "write () "is a job and "completed()" callback means the > job is done. > >> > If write() completed with less bytes processed than requested, it > >> should > >> > be > >> > called again. > >> > write() in TCP means data accepted for the trasnmission, but it does > >> not > >> > mean data is delivered. > >> > It is up to decoder to buffer encoded data or to push them into > >> internal > >> > channel. > >> > So shutdownOutput() works like flush and prohibits any future write > >> > operations. > >> > If error occurs during shutdownOutput() , exception should be thrown. > >> > If you are still worried about all data are transmitted on internal > >> > channel, > >> > you can wait in external channel close() the completion of all > >> operations > >> > on internal channel. > >> > > >> > >> Hi Alex, > >> > >> To be honest I don't fully understand the implications of > >> CharsetEncoder.encode(endOfInput=true) and CharsetEncoder.flush(). If I > >> write() 5 characters with endOfInput = false, > >> > >> 1) What's the point of CharsetEncoder.flush() if write(endOfInput=true) > >> already knows that you're flushing? Is it because flush() is > >> overridable by > >> subclasses whereas encode() is not? > >> 2) Am I guaranteed that the total characters outputted by > >> CharsetEncoder.write() and flush() will be equal to 5 characters? I ask > >> because the specification for flush() makes it seem as if there > >> is no limit > >> to the number of bytes it could output. > >> 3) If I want to implement a method similar to > >> OutputStream.flush() on top of > >> CharsetEncoder, how would I do it? write(endOfInput=true) and > >> flush() cannot > >> be invoked more than once, whereas OutputStream.flush() can. But > >> if I don't > >> invoke them am I really flushing? > >> > >> > >> Alexander Libman wrote: > >> > > >> > Yes, I know - it is still a draft, but it is working draft > and I hope > >> > the > >> > code is not hard to read. > >> > We will definetely try to find time to write better comments/doc. > >> > > >> > >> Okay. I will try to finalize my implementation in the coming > weeks. Once > >> that's done we'll discuss folding it into nio2 or your framework. I'd > >> like > >> to hand maintenance over to someone else once I finish the stabilizing > >> the > >> code. > >> > >> Gili > >> -- > >> View this message in context: > >> http://n2.nabble.com/AsynchronousSocketChannel.shutdownInput%28%29 > > -tp3365782p3388926.html > > Sent from the nio-discuss mailing list archive at Nabble.com. > > No virus found in this incoming message. > > Checked by AVG - www.avg.com > > Version: 8.5.392 / Virus Database: 270.13.44/2282 - Release > Date: 08/04/09 > > 18:01:00 > > > > > > > > -- > View this message in context: > http://n2.nabble.com/AsynchronousSocketChannel.shutdownInput%28%29 -tp3365782p3398196.html Sent from the nio-discuss mailing list archive at Nabble.com. No virus found in this incoming message. Checked by AVG - www.avg.com Version: 8.5.392 / Virus Database: 270.13.48/2291 - Release Date: 08/09/09 18:10:00 From vbonfanti at gmail.com Sun Aug 23 05:49:57 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Sun, 23 Aug 2009 08:49:57 -0400 Subject: file locking questions Message-ID: <18fbabce0908230549gb19ce38u1a6e2d52026ca1ed@mail.gmail.com> I'm working on a virtual file system for Google App Engine (http://code.google.com/p/gaevfs/) and my current implementation is based on Apache Commons VFS (http://commons.apache.org/vfs/). I was very excited to discover the NIO2 project and would like to adapt GaeVFS to use NIO2 as much as possible, but I have some questions related to file locking: 1 ) What, if any, are the expectations related to locking by the file system when a file is opened for reading and/or writing? Is the file system expected to do any automatic locking, or is the user expected to do all locking explicitly? Is this behavior file system dependent? 2) Why is it that SeekableFileChannel must be cast to FileChannel to get access to file locking? This implies that support for locking is optional or file system dependent--is that the intention? Commons VFS currently enforces that only a single output stream can be open for writing to a file. It does not implement any restrictions on the number of open input streams, and allows input streams to be opened while an output stream is already opened (and vice-versa). It also does not place any restrictions at all on opening a file for read/write random access, and does not provide any API for user-controlled file locking. (This all seems very inadequate). I thought I read somewhere in the JDK API docs that automatic locking (or not) is file system dependent; unfortunately, I can't seem to find that comment anywhere. If locking behavior is file system dependent, I'm thinking about implementing a scheme with the following characteristics: - automatically support a "one writer, many readers" locking scheme - opening a file for writing, either by getting an OutputStream or SeekableByteChannel with write access, gets an exclusive lock that blocks all other readers and writers - opening a file for reading, either by getting an InputStream or SeekableByteChannel with read access, gets a shared lock that blocks any writers but allows other readers - setting the "read-only" attribute for a file blocks all writers and allows all readers without requiring any locks - support an "no lock" option when opening a file for read and/or write access that would allow the user to control locking explicitly Does this seem like a a reasonable scheme? Or does this violate any conventions that I should be following (but can't seem to find documented anywhere)? Thanks. From Alan.Bateman at Sun.COM Sun Aug 23 09:18:59 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sun, 23 Aug 2009 17:18:59 +0100 Subject: file locking questions In-Reply-To: <18fbabce0908230549gb19ce38u1a6e2d52026ca1ed@mail.gmail.com> References: <18fbabce0908230549gb19ce38u1a6e2d52026ca1ed@mail.gmail.com> Message-ID: <4A916BF3.7090908@sun.com> Vince Bonfanti wrote: > : > > 1 ) What, if any, are the expectations related to locking by the file > system when a file is opened for reading and/or writing? Is the file > system expected to do any automatic locking, or is the user expected > to do all locking explicitly? Is this behavior file system dependent? > Yes, it is provider specific and there is no requirement to do automatic locking. If you have control over this then you might consider implementing it without any automatic locking or else support provider-specific OpenOptions to allow the application control the locking/sharing mode where needed. Our default provider on Windows does this to allow for interoperability, where required, with native programs that use legacy DOS sharing. If you don't have automatic locking then it allows several applications to open the same file for read/write and to use the file locking API to coordinate access (important if a file is used as a database for example). > 2) Why is it that SeekableFileChannel must be cast to FileChannel to > get access to file locking? This implies that support for locking is > optional or file system dependent--is that the intention? > Yes, this is intentional and meant for the cases where advanced features (like memory mapped I/O, positional read/write, or file locking) are required. The assumption is that these advanced features can't be feasibly implemented by all providers. One other thing to point out is that if an application really wants a FileChannel then the simple way is not to cast the SeekableByteChannel to a FileChannel but to instead invoke the FileChannel.open method to open the file. This delegates to the provider to create the channel (which is allowed to throw UOE if the provider does not support file channels). > : > > I thought I read somewhere in the JDK API docs that automatic locking > (or not) is file system dependent; unfortunately, I can't seem to find > that comment anywhere. > There isn't anything in the javadoc on this. You may be thinking of the "Platform Dependencies" section in the FileLock spec where it explains that the file locking API may be implemented on systems that use mandatory or advisory locking. > If locking behavior is file system dependent, I'm thinking about > implementing a scheme with the following characteristics: > > - automatically support a "one writer, many readers" locking scheme > - opening a file for writing, either by getting an OutputStream or > SeekableByteChannel with write access, gets an exclusive lock that > blocks all other readers and writers > - opening a file for reading, either by getting an InputStream or > SeekableByteChannel with read access, gets a shared lock that blocks > any writers but allows other readers > - setting the "read-only" attribute for a file blocks all writers and > allows all readers without requiring any locks > - support an "no lock" option when opening a file for read and/or > write access that would allow the user to control locking explicitly > > Does this seem like a a reasonable scheme? Or does this violate any > conventions that I should be following (but can't seem to find > documented anywhere)? > This seems reasonable, assuming that blocking other programs means that attempts to open the file with a conflicting mode will cause a reasonable I/O exception to be thrown. Note that our Windows provider can be made to work in this way by specifying the provider-specific NOSHARE_WRITE option when opening a file for reading, and the NOSHARE_READ and NOSHARE_WRITE options when opening a file for writing. In addition to considering other reader and writers, you'll also need to think about other operations (delete, moveTo, copyTo, etc. while the file is in use). Hopefully this helps. Please keep us updated on the issues you run into as you develop this provider. -Alan. From avinash.lakshman at gmail.com Sun Aug 23 09:28:07 2009 From: avinash.lakshman at gmail.com (Avinash Lakshman) Date: Sun, 23 Aug 2009 09:28:07 -0700 Subject: file locking questions In-Reply-To: <4A916BF3.7090908@sun.com> References: <18fbabce0908230549gb19ce38u1a6e2d52026ca1ed@mail.gmail.com> <4A916BF3.7090908@sun.com> Message-ID: Maybe this is slightly off topic but wouldn't the addition of large array support be great here. I am assuming that RAM file system in the VFS can only use files that can get as large as 2 GB since is is maintained using some sort of an array addressed using an int. If the machine has large amount of memory I am assuming that one would need to maintain multiple 2 GB file systems in order to use the memory more effectively. Wouldn't this be a good use case for providing large array support so that RAM file system could hold files as large as permissible memory? I am new to this VFS thingy so if I am way off please correct me. Avinash On Sun, Aug 23, 2009 at 9:18 AM, Alan Bateman wrote: > Vince Bonfanti wrote: > >> : >> >> 1 ) What, if any, are the expectations related to locking by the file >> system when a file is opened for reading and/or writing? Is the file >> system expected to do any automatic locking, or is the user expected >> to do all locking explicitly? Is this behavior file system dependent? >> >> > Yes, it is provider specific and there is no requirement to do automatic > locking. If you have control over this then you might consider implementing > it without any automatic locking or else support provider-specific > OpenOptions to allow the application control the locking/sharing mode where > needed. Our default provider on Windows does this to allow for > interoperability, where required, with native programs that use legacy DOS > sharing. If you don't have automatic locking then it allows several > applications to open the same file for read/write and to use the file > locking API to coordinate access (important if a file is used as a database > for example). > > 2) Why is it that SeekableFileChannel must be cast to FileChannel to >> get access to file locking? This implies that support for locking is >> optional or file system dependent--is that the intention? >> >> > Yes, this is intentional and meant for the cases where advanced features > (like memory mapped I/O, positional read/write, or file locking) are > required. The assumption is that these advanced features can't be feasibly > implemented by all providers. > > One other thing to point out is that if an application really wants a > FileChannel then the simple way is not to cast the SeekableByteChannel to a > FileChannel but to instead invoke the FileChannel.open method to open the > file. This delegates to the provider to create the channel (which is allowed > to throw UOE if the provider does not support file channels). > > : >> >> I thought I read somewhere in the JDK API docs that automatic locking >> (or not) is file system dependent; unfortunately, I can't seem to find >> that comment anywhere. >> >> > There isn't anything in the javadoc on this. You may be thinking of the > "Platform Dependencies" section in the FileLock spec where it explains that > the file locking API may be implemented on systems that use mandatory or > advisory locking. > > If locking behavior is file system dependent, I'm thinking about >> implementing a scheme with the following characteristics: >> >> - automatically support a "one writer, many readers" locking scheme >> - opening a file for writing, either by getting an OutputStream or >> SeekableByteChannel with write access, gets an exclusive lock that >> blocks all other readers and writers >> - opening a file for reading, either by getting an InputStream or >> SeekableByteChannel with read access, gets a shared lock that blocks >> any writers but allows other readers >> - setting the "read-only" attribute for a file blocks all writers and >> allows all readers without requiring any locks >> - support an "no lock" option when opening a file for read and/or >> write access that would allow the user to control locking explicitly >> >> Does this seem like a a reasonable scheme? Or does this violate any >> conventions that I should be following (but can't seem to find >> documented anywhere)? >> >> > This seems reasonable, assuming that blocking other programs means that > attempts to open the file with a conflicting mode will cause a reasonable > I/O exception to be thrown. Note that our Windows provider can be made to > work in this way by specifying the provider-specific NOSHARE_WRITE option > when opening a file for reading, and the NOSHARE_READ and NOSHARE_WRITE > options when opening a file for writing. In addition to considering other > reader and writers, you'll also need to think about other operations > (delete, moveTo, copyTo, etc. while the file is in use). > > Hopefully this helps. Please keep us updated on the issues you run into as > you develop this provider. > > -Alan. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090823/482032fe/attachment.html From Alan.Bateman at Sun.COM Sun Aug 23 09:40:47 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sun, 23 Aug 2009 17:40:47 +0100 Subject: file locking questions In-Reply-To: References: <18fbabce0908230549gb19ce38u1a6e2d52026ca1ed@mail.gmail.com> <4A916BF3.7090908@sun.com> Message-ID: <4A91710F.2000707@sun.com> Avinash Lakshman wrote: > Maybe this is slightly off topic but wouldn't the addition of large > array support be great here. I am assuming that RAM file system in the > VFS can only use files that can get as large as 2 GB since is is > maintained using some sort of an array addressed using an int. If the > machine has large amount of memory I am assuming that one would need > to maintain multiple 2 GB file systems in order to use the memory more > effectively. Wouldn't this be a good use case for providing large > array support so that RAM file system could hold files as large as > permissible memory? I am new to this VFS thingy so if I am way off > please correct me. It depends on how the provider chooses to implement the underlying storage but usually it will be in blocks. For example, a block might be a 4k direct buffer. Blocks are allocated to files as they grow and returned to the some pool when files are truncated or deleted. -Alan. From vbonfanti at gmail.com Sun Aug 23 17:34:22 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Sun, 23 Aug 2009 20:34:22 -0400 Subject: FileSystemProvider.getFileSystem() handling of IOException thrown by newFileSystem()? Message-ID: <18fbabce0908231734o636756b7rf9c2288ccd7937de@mail.gmail.com> The JavaDocs for java.nio.file.spi.FileSystemProvider (http://java.sun.com/javase/7/docs/api/java/nio/file/spi/FileSystemProvider.html) make it clear that getFileSystem() is supposed to invoke newFileSystem() to create a new FileSystem instance, but...newFileSystem() declares that it throws IOException, but getFileSystem() does not declare any thrown exceptions (all of the exceptions listed in the JavaDocs are subclasses of RuntimeException). What is getFileSystem() supposed to do if newFileSystem() throws an IOException? Wrap the IOException in a RuntimeException and rethrow it? return null? something else? Is there any documentation available for authors of file system providers other than the JavaDocs and resources listed on the project web site (http://openjdk.java.net/projects/nio/)? Thanks. From Alan.Bateman at Sun.COM Mon Aug 24 01:15:31 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 24 Aug 2009 09:15:31 +0100 Subject: FileSystemProvider.getFileSystem() handling of IOException thrown by newFileSystem()? In-Reply-To: <18fbabce0908231734o636756b7rf9c2288ccd7937de@mail.gmail.com> References: <18fbabce0908231734o636756b7rf9c2288ccd7937de@mail.gmail.com> Message-ID: <4A924C23.20300@sun.com> Vince Bonfanti wrote: > The JavaDocs for java.nio.file.spi.FileSystemProvider > (http://java.sun.com/javase/7/docs/api/java/nio/file/spi/FileSystemProvider.html) > make it clear that getFileSystem() is supposed to invoke > newFileSystem() to create a new FileSystem instance, > The intention is that getFileSystem returns a reference to an existing FileSystem (that was created previously by calling the provider's newFileSystem method). So getFileSystem doesn't create a FileSystem - instead it throws the unchecked FileSystemNotFoundException if it does not exist. I think we improve the wording of the method description to make this clearer. > : > > Is there any documentation available for authors of file system > providers other than the JavaDocs and resources listed on the project > web site (http://openjdk.java.net/projects/nio/)? > We don't, but if you keep notes on the issues you run into then perhaps we could add a page to aid future provider implementers. In the demo directory you'll find a prototype of a zip provider (written by Rajendra Gutupalli) that might be useful. -Alan. From vbonfanti at gmail.com Mon Aug 24 08:27:02 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Mon, 24 Aug 2009 11:27:02 -0400 Subject: FileSystemProvider.getFileSystem() handling of IOException thrown by newFileSystem()? In-Reply-To: <4A924C23.20300@sun.com> References: <18fbabce0908231734o636756b7rf9c2288ccd7937de@mail.gmail.com> <4A924C23.20300@sun.com> Message-ID: <18fbabce0908240827r209ad95bgf7559c3b43a82e3f@mail.gmail.com> On Mon, Aug 24, 2009 at 4:15 AM, Alan Bateman wrote: > Vince Bonfanti wrote: >> >> The JavaDocs for java.nio.file.spi.FileSystemProvider >> >> (http://java.sun.com/javase/7/docs/api/java/nio/file/spi/FileSystemProvider.html) >> make it clear that getFileSystem() is supposed to invoke >> newFileSystem() to create a new FileSystem instance, >> > > The intention is that getFileSystem returns a reference to an existing > FileSystem (that was created previously by calling the provider's > newFileSystem method). So getFileSystem doesn't create a FileSystem - > instead it throws the unchecked FileSystemNotFoundException if it does not > exist. I think we improve the wording of the method description to make this > clearer. > OK. I'm overriding the default provider and in this case it seems that my provider is expected to create the default file system automatically (without a call to newFileSystem), which makes sense. >> : >> >> Is there any documentation available for authors of file system >> providers other than the JavaDocs and resources listed on the project >> web site (http://openjdk.java.net/projects/nio/)? >> > > We don't, but if you keep notes on the issues you run into then perhaps we > could add a page to aid future provider implementers. In the demo directory > you'll find a prototype of a zip provider (written by Rajendra Gutupalli) > that might be useful. > Perfect. Thanks. Vince From Alan.Bateman at Sun.COM Mon Aug 24 09:40:10 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 24 Aug 2009 17:40:10 +0100 Subject: FileSystemProvider.getFileSystem() handling of IOException thrown by newFileSystem()? In-Reply-To: <18fbabce0908240827r209ad95bgf7559c3b43a82e3f@mail.gmail.com> References: <18fbabce0908231734o636756b7rf9c2288ccd7937de@mail.gmail.com> <4A924C23.20300@sun.com> <18fbabce0908240827r209ad95bgf7559c3b43a82e3f@mail.gmail.com> Message-ID: <4A92C26A.9020209@sun.com> Vince Bonfanti wrote: > : > OK. I'm overriding the default provider and in this case it seems that > my provider is expected to create the default file system > automatically (without a call to newFileSystem), which makes sense. > Ah, I didn't realize you were replacing the default provider (are you sure this is what you want?). In that case, I see we have an inconsistency in the spec in that it does say (in FileSystems.getDefault) that the provider's getFileSystem is invoked to "create" the default file system. It does of course, just invoke getFileSystem to get a reference to the default file system (which is created automatically at a time of the provider's choosing). I'll create a bug and get this fixed - sorry I missed this when we exchanged mail yesterday as I thought you were creating a provider for a pseudo file system. As you are replacing the default provider then one thing I should point is out that file I/O using java.io will not go through your provider, not yet anyway. The reason is that we haven't pushed the changes for this yet as there are a few subtle issues to figure out. I just mention this in case this was your goal. Also, I mention it so that you are aware of potential bootstrapping issues if your provider requires file I/O during its initialization. If it does, then you'll need to delegate to the system default provider during the initialization (your provider's constructor will be called with a reference to this provider). -Alan. From vbonfanti at gmail.com Mon Aug 24 13:40:12 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Mon, 24 Aug 2009 16:40:12 -0400 Subject: java.nio.file.spi.AbstractPath? Message-ID: <18fbabce0908241340r3c0b63ccxc9f876947f611783@mail.gmail.com> The JavaDocs list an AbstractPath class, but I can't find an implementation anywhere: http://java.sun.com/javase/7/docs/api/java/nio/file/spi/AbstractPath.html I've downloaded the May 19, 2009 binary snapshot (b101?) from the NIO2 project page, and also the August 20, 2009 source bundle (b70?) from the OpenJDK download page, but neither contains AbstractPath. BTW, which of these is the latest? In general, what's the best way to get the latest source (other than via Mercurial--I have no experience with Mercurial--and I'm not sure I need to be that close to the bleeding edge anyway). Thanks. Vince From Alan.Bateman at Sun.COM Mon Aug 24 14:58:40 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 24 Aug 2009 22:58:40 +0100 Subject: java.nio.file.spi.AbstractPath? In-Reply-To: <18fbabce0908241340r3c0b63ccxc9f876947f611783@mail.gmail.com> References: <18fbabce0908241340r3c0b63ccxc9f876947f611783@mail.gmail.com> Message-ID: <4A930D10.6000803@sun.com> Vince Bonfanti wrote: > The JavaDocs list an AbstractPath class, but I can't find an > implementation anywhere: > > http://java.sun.com/javase/7/docs/api/java/nio/file/spi/AbstractPath.html > > I've downloaded the May 19, 2009 binary snapshot (b101?) from the NIO2 > project page, and also the August 20, 2009 source bundle (b70?) from > the OpenJDK download page, but neither contains AbstractPath. > > BTW, which of these is the latest? In general, what's the best way to > get the latest source (other than via Mercurial--I have no experience > with Mercurial--and I'm not sure I need to be that close to the > bleeding edge anyway). Thanks. > > Vince > There are weekly builds of jdk7 with periodic milestone releases. At this time, the current build is b70 with milestone 5 coming at b73. You can find out more at the jdk7 project page (see the links to builds and milestones): http://openjdk.java.net/projects/jdk7/ For what you are doing, I would suggest tracking the weekly builds. The binaries and javadoc can be found here: http://download.java.net/jdk7/binaries/ http://openjdk.java.net/projects/jdk7/builds/ The java.sun.com site serves up the milestone releases, the latest being m4 (b66). Unfortunately it seems the online javadoc hasn't been updated and it rather stale javadoc from b59. This is why you are seeing AbstractPath even though it is gone a long time. Sorry about that. I've just mailed the person that owns this to see if it can be fixed. As regards the early access build from the nio project page - these were builds to allow folks try out the API and changes before going into jdk7. We haven't done a build available since May because everything is in jdk7 and all the fixes are going direct to jdk7. It's on my list to update the project page and direct readers to the jdk7 weekly builds. The best way to get the latest sources is to clone the http://hg.openjdk.java.net/jdk7/jdk7 forest (Mercurial is relatively simple to use). You can browse the tip here: http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/tip/ -Alan. From vbonfanti at gmail.com Tue Aug 25 11:33:27 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Tue, 25 Aug 2009 14:33:27 -0400 Subject: Path.createFile() and Path.createDirectory() throw NoSuchFileException if parent doesn't exist (Windows) Message-ID: <18fbabce0908251133x749bf68fj7d6984d9baa40b02@mail.gmail.com> I notice that on Windows both Path.createFile() and Path.createDirectory() throw a NoSuchFileException if their parent directory doesn't exist. Is this expected behavior for all file systems, or is it optional for a file system to create the parent directories if they don't exist? In either case, it should documented in the Javadocs. Vince From Alan.Bateman at Sun.COM Tue Aug 25 11:59:15 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 25 Aug 2009 19:59:15 +0100 Subject: Path.createFile() and Path.createDirectory() throw NoSuchFileException if parent doesn't exist (Windows) In-Reply-To: <18fbabce0908251133x749bf68fj7d6984d9baa40b02@mail.gmail.com> References: <18fbabce0908251133x749bf68fj7d6984d9baa40b02@mail.gmail.com> Message-ID: <4A943483.9050504@sun.com> Vince Bonfanti wrote: > I notice that on Windows both Path.createFile() and > Path.createDirectory() throw a NoSuchFileException if their parent > directory doesn't exist. Is this expected behavior for all file > systems, or is it optional for a file system to create the parent > directories if they don't exist? In either case, it should documented > in the Javadocs. > > Vince > All of Path's createXXX method are specified to throw IOException if they fail (a non-existent parent directory being one of many possible reasons). Implementations can throw more specific exceptions where they can detect specific errors and map them to a specific IOException. So if the parent directory doesn't exist then the createXXX methods should fail but it's up to the implementation if it throws more specific/useful IOException or not. In this case, we attempt to map this case to NoSuchFileException. Does that help? Where you need to create a file tree then the Files.createDirectories utility method can be used. -Alan. From vbonfanti at gmail.com Tue Aug 25 12:18:01 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Tue, 25 Aug 2009 15:18:01 -0400 Subject: Path.createFile() and Path.createDirectory() throw NoSuchFileException if parent doesn't exist (Windows) In-Reply-To: <4A943483.9050504@sun.com> References: <18fbabce0908251133x749bf68fj7d6984d9baa40b02@mail.gmail.com> <4A943483.9050504@sun.com> Message-ID: <18fbabce0908251218o19b7a72em715993433e165267@mail.gmail.com> Yes, that helps. Thanks. It may be a minor point, but I suggest that the Javadocs for Path explicitly state that an exception is thrown if the parent doesn't exist; maybe even list NoSuchFileException as an "optional specific exception" as is done for FileAlreadyExistsException. Because the Javadocs for Path are silent on this point, I wasn't sure about the behavior until I actually tested it. Vince On Tue, Aug 25, 2009 at 2:59 PM, Alan Bateman wrote: > Vince Bonfanti wrote: >> >> I notice that on Windows both Path.createFile() and >> Path.createDirectory() throw a NoSuchFileException if their parent >> directory doesn't exist. Is this expected behavior for all file >> systems, or is it optional for a file system to create the parent >> directories if they don't exist? In either case, it should documented >> in the Javadocs. >> >> Vince >> > > All of Path's createXXX method are specified to throw IOException if they > fail (a non-existent parent directory being one of many possible reasons). > ?Implementations can throw more specific exceptions where they can detect > specific errors and map them to a specific IOException. So if the parent > directory doesn't exist then the createXXX methods should fail but it's up > to the implementation if it throws more specific/useful IOException or not. > In this case, we attempt to map this case to NoSuchFileException. Does that > help? Where you need to create a file tree then the Files.createDirectories > utility method can be used. > > -Alan. > From Alan.Bateman at Sun.COM Tue Aug 25 12:43:01 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 25 Aug 2009 20:43:01 +0100 Subject: Path.createFile() and Path.createDirectory() throw NoSuchFileException if parent doesn't exist (Windows) In-Reply-To: <18fbabce0908251218o19b7a72em715993433e165267@mail.gmail.com> References: <18fbabce0908251133x749bf68fj7d6984d9baa40b02@mail.gmail.com> <4A943483.9050504@sun.com> <18fbabce0908251218o19b7a72em715993433e165267@mail.gmail.com> Message-ID: <4A943EC5.5000809@sun.com> Vince Bonfanti wrote: > Yes, that helps. Thanks. > > It may be a minor point, but I suggest that the Javadocs for Path > explicitly state that an exception is thrown if the parent doesn't > exist; maybe even list NoSuchFileException as an "optional specific > exception" as is done for FileAlreadyExistsException. Because the > Javadocs for Path are silent on this point, I wasn't sure about the > behavior until I actually tested it. > > Vince > Okay, we can make this clearer and explicitly state that it doesn't attempt to create parent directories (need to think more about listing NoSuchFileException as a specific exception). -Alan. From mthornton at optrak.co.uk Thu Aug 27 07:37:35 2009 From: mthornton at optrak.co.uk (Mark Thornton) Date: Thu, 27 Aug 2009 15:37:35 +0100 Subject: Should FileTime be serializable? Message-ID: <4A969A2F.6020508@optrak.co.uk> That is java.nio.file.attribute.FileTime. According to the b70 documentation it isn't. Regards, Mark Thornton From cowwoc at bbs.darktech.org Thu Aug 27 09:21:37 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 27 Aug 2009 11:21:37 -0500 (CDT) Subject: What happens if a Channel is read after being closed? Message-ID: <1251390097141-3527766.post@n2.nabble.com> The Javadoc talks about what happens if a channel is closed while an operation is pending but it says nothing about what happens if someone tries reading after a close() operation completed. Shouldn't the methods throw an appropriate exception? Gili -- View this message in context: http://n2.nabble.com/What-happens-if-a-Channel-is-read-after-being-closed-tp3527766p3527766.html Sent from the nio-discuss mailing list archive at Nabble.com. From cowwoc at bbs.darktech.org Thu Aug 27 09:43:41 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 27 Aug 2009 11:43:41 -0500 (CDT) Subject: What happens if a Channel is read after being closed? In-Reply-To: <1251390097141-3527766.post@n2.nabble.com> References: <1251390097141-3527766.post@n2.nabble.com> Message-ID: <1251391421516-3528051.post@n2.nabble.com> Oops, AsynchronousChannel.close()'s Javadoc reads: "further attempts to initiate asynchronous I/O operations complete immediately with cause ClosedChannelException." That being said, shouldn't the read/write methods still document this as an exception that may be thrown? Gili Gili wrote: > > The Javadoc talks about what happens if a channel is closed while an > operation is pending but it says nothing about what happens if someone > tries reading after a close() operation completed. Shouldn't the methods > throw an appropriate exception? > > Gili > -- View this message in context: http://n2.nabble.com/What-happens-if-a-Channel-is-read-after-being-closed-tp3527766p3528051.html Sent from the nio-discuss mailing list archive at Nabble.com. From Alan.Bateman at Sun.COM Thu Aug 27 10:26:25 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 27 Aug 2009 18:26:25 +0100 Subject: Should FileTime be serializable? In-Reply-To: <4A969A2F.6020508@optrak.co.uk> References: <4A969A2F.6020508@optrak.co.uk> Message-ID: <4A96C1C1.3000802@sun.com> Mark Thornton wrote: > That is java.nio.file.attribute.FileTime. According to the b70 > documentation it isn't. > > Regards, > Mark Thornton > The class does lend itself to being serialized. Have you come across a need for this? -Alan. From Alan.Bateman at Sun.COM Thu Aug 27 10:31:35 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 27 Aug 2009 18:31:35 +0100 Subject: What happens if a Channel is read after being closed? In-Reply-To: <1251391421516-3528051.post@n2.nabble.com> References: <1251390097141-3527766.post@n2.nabble.com> <1251391421516-3528051.post@n2.nabble.com> Message-ID: <4A96C2F7.2030205@sun.com> Gili wrote: > Oops, AsynchronousChannel.close()'s Javadoc reads: > > "further attempts to initiate asynchronous I/O operations complete > immediately with cause ClosedChannelException." > > That being said, shouldn't the read/write methods still document this as an > exception that may be thrown? > > The read/write methods don't throw I/O exceptions or any checked exception, at least not directly. Instead the I/O operation completes immediately with ClosedChannelException. So where you use a CompletionHandler, its failed method is invoked with the ClosedChannelException. If using a Future then its get method throws ExecutionException with ClosedChannelException as the cause. I agree this isn't obvious from just looking at the specific I/O methods so I'll look into how we might improve this. -Alan From vbonfanti at gmail.com Thu Aug 27 19:30:26 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Thu, 27 Aug 2009 22:30:26 -0400 Subject: file locking questions In-Reply-To: <4A916BF3.7090908@sun.com> References: <18fbabce0908230549gb19ce38u1a6e2d52026ca1ed@mail.gmail.com> <4A916BF3.7090908@sun.com> Message-ID: <18fbabce0908271930t5eae6a62yea16d002d7c0c5c6@mail.gmail.com> On Sun, Aug 23, 2009 at 12:18 PM, Alan Bateman wrote: > Vince Bonfanti wrote: >> >> : >> >> I thought I read somewhere in the JDK API docs that automatic locking >> (or not) is file system dependent; unfortunately, I can't seem to find >> that comment anywhere. >> > > There isn't anything in the javadoc on this. You may be thinking of the > "Platform Dependencies" section in the FileLock spec where it explains that > the file locking API may be implemented on systems that use mandatory or > advisory locking. > In case anyone's interested, I found the reference I was thinking about, for java.io.FileOutputStream: "Some platforms, in particular, allow a file to be opened for writing by only one FileOutputStream (or other file-writing object) at a time. In such situations the constructors in this class will fail if the file involved is already open." Further, all of the FileOutputStream constructors say this: "Throws: FileNotFoundException - if the file exists but is a directory rather than a regular file, does not exist but cannot be created, or cannot be opened for any other reason." This indicates that on some platforms a FileOutputStream constructor might throw a FileNotFoundException if the file exists, but is opened for writing by another thread. From mthornton at optrak.co.uk Fri Aug 28 01:26:07 2009 From: mthornton at optrak.co.uk (Mark Thornton) Date: Fri, 28 Aug 2009 09:26:07 +0100 Subject: Should FileTime be serializable? In-Reply-To: <4A96C1C1.3000802@sun.com> References: <4A969A2F.6020508@optrak.co.uk> <4A96C1C1.3000802@sun.com> Message-ID: <4A97949F.7060400@optrak.co.uk> Alan Bateman wrote: > Mark Thornton wrote: >> That is java.nio.file.attribute.FileTime. According to the b70 >> documentation it isn't. >> >> Regards, >> Mark Thornton >> > The class does lend itself to being serialized. Have you come across a > need for this? > > -Alan. > I was contemplating a simple backup application to test out the new api and decided to serialize the file metadata. Some it (e.g. UserPrincipal) needs to be translated to another form, but I did not see any good reason why FileTime could not serialize itself. Mark From vbonfanti at gmail.com Fri Aug 28 11:30:25 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Fri, 28 Aug 2009 14:30:25 -0400 Subject: suggestion for convenience method for BasicFileAttributes Message-ID: <18fbabce0908281130n57791e59u70d74ad770933e58@mail.gmail.com> A minor suggestion: as a convenience for provider implementors, it seems that BasicFileAttributes could be declared abstract (rather than an interface), and implement a method that looks something like below. Otherwise, it seems that every provider implementor is going to have to implement their own method that does exactly this (or something similar). Vince public Object readAttribute( String attrName ) { if ( "lastModifiedTime".equals( attrName ) ) { return lastModifiedTime(); } else if ( "lastAccessTime".equals( attrName ) ) { return lastAccessTime(); } else if ( "creationTime".equals( attrName ) ) { return creationTime(); } else if ( "size".equals( attrName ) ) { return size(); } else if ( "isRegularFile".equals( attrName ) ) { return isRegularFile(); } else if ( "isDirectory".equals( attrName ) ) { return isDirectory(); } else if ( "isSymbolicLink".equals( attrName ) ) { return isSymbolicLink(); } else if ( "isOther".equals( attrName ) ) { return isOther(); } else if ( "fileKey".equals( attrName ) ) { return fileKey(); } else { return null; } } From cowwoc at bbs.darktech.org Fri Aug 28 21:05:27 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Fri, 28 Aug 2009 23:05:27 -0500 (CDT) Subject: AsynchronousByteCharChannel and Timeouts In-Reply-To: <4A7A9D4F.8000408@sun.com> References: <4A74B137.9020204@sun.com> <1249163959897-3370772.post@n2.nabble.com> <4A75F214.6080105@sun.com> <1249320227257-3379223.post@n2.nabble.com> <4A77453E.5090103@sun.com> <4A78F73B.8060900@bbs.darktech.org> <4A793F85.3060907@sun.com> <1249481863674-3392148.post@n2.nabble.com> <4A7A9D4F.8000408@sun.com> Message-ID: <1251518727842-3540435.post@n2.nabble.com> Hi Alan, Replies below... Alan Bateman wrote: > > Gili wrote: >> : >> Okay. I agree with everything you wrote except for the earlier post about >> how it should be possible to implement read-with-timeout in terms of >> read-forever. > Right, this is where we differ because AsynchronousSocketChannel does > not require the implementation to have a mechanism to cancel the > underlying I/O operation. > I've got an idea I want to run by you. I propose defining read/write with timeout at the AsynchronousByteChannel level because it'll simplify the class hierarchy (no need to define separate helper classes for read-with-timeout versus read-without-timeout). I believe that these methods can be implemented across all platforms in the following way: Case 1: read(forever), followed by Future.cancel() \-> if the native layer supports read(timeout) then we emulate read(forever) in terms of a loop containing read(timeout) where timeout is a reasonably small value. The loop runs until data becomes available or until Future.cancel() is invoked. In the latter case, Future.cancel() blocks until the timeout completes and then return true. Any data read before Future.cancel() is invoked is buffered for subsequent reads. \-> if the native layer does not support read(timeout), Future.cancel() returns false and subsequent reads throw RuntimeException (similar to AsynchronousSocketChannel). Case 2: read(forever) with a CompletionHandler: the native read terminates when data becomes available or if the channel is closed. Case 3: read(timeout) with a CompletionHandler \-> if the native layer supports timeouts, the native read terminates when data becomes available, a timeout occurs or the channel is closed. \-> if the native layer does not support timeouts and a timeout occurs before the native read completes then subsequent reads throw RuntimeException. Case 4: write(forever), followed by Future.cancel() \-> if the native layer supports write(timeout) then we emulate write(forever) in terms of a loop containing write(timeout) where timeout is a reasonably small value. The loop runs until data becomes available or until Future.cancel() is invoked. In the latter case, Future.cancel() blocks until the timeout completes and then return true. If Future.cancel() is invoked before a single byte is written then no data is written, otherwise any data not yet sent is buffered to proceed subsequent writes. \-> if the native layer does not support write(timeout), Future.cancel() returns false and subsequent reads throw RuntimeException. Case 5: write(timeout) with a CompletionHandler: the native write terminates when the data is fully written or if the channel is closed. Case 6: write(timeout) with a CompletionHandler \-> if the native layer supports timeouts, the native write terminates when the data is fully written, a timeout occurs or the channel is closed. \-> if the native layer does not support timeouts and a timeout occurs before the native write completes then subsequent reads throw RuntimeException. Let me know what you think. Thanks, Gili -- View this message in context: http://n2.nabble.com/AsynchronousByteCharChannel-and-Timeouts-tp3363004p3540435.html Sent from the nio-discuss mailing list archive at Nabble.com. From vbonfanti at gmail.com Sat Aug 29 08:29:27 2009 From: vbonfanti at gmail.com (Vince Bonfanti) Date: Sat, 29 Aug 2009 11:29:27 -0400 Subject: questions about mandatory/optional file attributes Message-ID: <18fbabce0908290829y33afc617sd37da5cbfc9b9efc@mail.gmail.com> The BasicFileAttributes javadoc says, "Basic file attributes are attributes that are common to many file systems and consist of mandatory and optional file attributes as defined by this interface." The descriptions for lastModifiedTime(), lastAccessTime(), and creationTime() all say they return "null if the attribute is not supported" so it's clear that these three are optional. The description for fileKey() says it returns "null if a file key is not available" so it seems that's also optional. There's nothing that explicitly says which attributes are mandatory--does that mean all other attributes are mandatory? If so, here's my list: optional: lastModifiedTime(), lastAccessTime(), creationTime(), fileKey() mandatory: isRegularFile(), isDirectory(), isSymbolicLink(), isOther(), size() Question #1: Is this list of mandatory/optional attributes correct? Specifically, if my file system implementation does not support symbolic links, am I supposed to return a boolean for isSymbolicLink(), rather than throwing UnsupportedOperationException? The FileRef.getAttribute() javadoc says it returns "null if ... it does not support reading the attribute." The FileRef.readAttributes() javadoc says, "Attributes that are not supported are ignored and will not be present in the returned map." Question #2: Does "supported" in the FileRef javadocs take the same meaning as "mandatory" on the BasisFileAttributes javadocs? Specifically, if my file system implementation does not support symbolic links, am I supposed to return a Boolean from FileRef.getAttribute( "isSymbolicLink" ), rather than returning null? Finally, it appears there are no direct equivalents to java.io.File.isDirectory(), isFile(), lastModified() or length() in the java.nio.file.Path class. For example, the two most direct ways I can find in NIO2 to do the equivalent of java.io.File.isDirectory() are: Attributes.readBasicFileAttributes( myPath ).isDirectory(); ((Boolean)myPath.readAttribute( "isDirectory" )).booleanValue(); These both strike me as awkward and a step backwards from java.io.File. Question #3: If mandatory attributes are truly mandatory, why not provide direct type-safe accessors from the FileRef interface? After all, if FileRef.readAttribute( "isDirectory" ) is required to return a non-null (and, presumably, Boolean) value, why not define FileRef.isDirectory() to return a boolean? From Alan.Bateman at Sun.COM Mon Aug 31 22:58:47 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 01 Sep 2009 06:58:47 +0100 Subject: java.nio.file.spi.AbstractPath? In-Reply-To: <4A930D10.6000803@sun.com> References: <18fbabce0908241340r3c0b63ccxc9f876947f611783@mail.gmail.com> <4A930D10.6000803@sun.com> Message-ID: <4A9CB817.9030701@sun.com> Alan Bateman wrote: > : > > The java.sun.com site serves up the milestone releases, the latest > being m4 (b66). Unfortunately it seems the online javadoc hasn't been > updated and it rather stale javadoc from b59. This has been fixed so that the binaries and javadoc on java.sun.com are in sync (with m4). This does seem to be a one-time glitch. -Alan.