From baier at ontoprise.de Wed Jun 10 15:59:56 2009 From: baier at ontoprise.de (=?iso-8859-1?Q?J=FCrgen_Baier?=) Date: Thu, 11 Jun 2009 00:59:56 +0200 Subject: BigByteBuffer/MappedBigByteBuffer Message-ID: <05A67406923D4BC2B44FFB85F01DFB45@grobi2> Hi, I just downloaded the latest JDK7 and I could not find the big buffer classes (BigByteBuffer, ...). Are the big buffers still on the roadmap? When can we expect a first version? I think these buffers are very important for dealing with very large memories (especially for data-intensive applications). Regards, Juergen -- Juergen Baier Research & Development ontoprise GmbH - know how to use Know-how ------ ontoprise presents the new SemanticMiner for SharePoint ------ Amalienbadstra?e 36 (Raumfabrik 29); 76227 Karlsruhe eMail: baier at ontoprise.de www: http://www.ontoprise.de ------ An der RaumFabrik 29; 76227 Karlsruhe; Germany mailto:baier at ontoprise.de, www: http://www.ontoprise.de Registered office: Karlsruhe, Germany; Register court: Mannheim, HRB 109540 Managing directors: Prof. Dr. J?rgen Angele, Dipl.Wi.-Ing. Hans-Peter Schnurr From Alan.Bateman at Sun.COM Thu Jun 11 04:48:05 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 11 Jun 2009 12:48:05 +0100 Subject: BigByteBuffer/MappedBigByteBuffer In-Reply-To: <05A67406923D4BC2B44FFB85F01DFB45@grobi2> References: <05A67406923D4BC2B44FFB85F01DFB45@grobi2> Message-ID: <4A30EEF5.5040603@sun.com> J?rgen Baier wrote: > Hi, > > I just downloaded the latest JDK7 and I could not find the big buffer > classes (BigByteBuffer, ...). Are the big buffers still on the roadmap? When > can we expect a first version? > > I think these buffers are very important for dealing with very large > memories (especially for data-intensive applications). > > Regards, > > Juergen > The original early draft did propose a set of 64-bit addressable buffers but we aren't taking that proposal any further. The real demand is for 64-bit arrays or collections. For I/O, the use-case is contiguous mapping of file regions larger than 2GB. At one point Doug Lea, and the collections group looked into creating a package of big arrays for each of the scaler types. With such a solution we could do something to allow the array be backed by a file mapping. More recently, there was a proposal for large arrays [1]. This one is on the list for "further consideration" [2] by the COIN project. Can you say any more about your needs? -Alan. [1] http://mail.openjdk.java.net/pipermail/coin-dev/2009-March/000869.html [2] http://blogs.sun.com/darcy/entry/project_coin_consideration_round_2 From baier at ontoprise.de Thu Jun 11 06:54:36 2009 From: baier at ontoprise.de (=?iso-8859-1?Q?J=FCrgen_Baier?=) Date: Thu, 11 Jun 2009 15:54:36 +0200 Subject: AW: BigByteBuffer/MappedBigByteBuffer In-Reply-To: <4A30EEF5.5040603@sun.com> References: <05A67406923D4BC2B44FFB85F01DFB45@grobi2> <4A30EEF5.5040603@sun.com> Message-ID: Hi Alan, thanks for the quick answer. > The original early draft did propose a set of 64-bit addressable buffers but > we aren't taking that proposal any further. The real demand is for 64-bit > arrays or collections. For I/O, the use-case is contiguous mapping of file > regions larger than 2GB. At one point Doug Lea, and the collections group > looked into creating a package of big arrays for each of the scaler types. > With such a solution we could do something to allow the array be backed by a > file mapping. More recently, there was a proposal for large arrays [1]. This > one is on the list for "further consideration" [2] by the COIN project. > > Can you say any more about your needs? There are two important requirements: 1. 64-bit buffers ----------------- If I have (e.g.) 12 GB RAM available then it should be possible to address more than 32-bit addresses. This is often required for simulations, mathematical software, software for bio informatics, main-memory databases etc. Without native support for large arrays/buffers developers have to implement some workaround (e.g. use multiple arrays of ByteBuffers/32-bit arrays or implement some native code). 64-bit arrays might be a good replacement for the big buffers. But (at least on my system) the direct byte buffer is much faster than the non-direct (heap) byte buffer. 2. Memory-mapped I/O -------------------- Contiguous mapping of files larger than 2 GB is very useful, too (for all kinds of applications which have to deal with files of this size). > At one point Doug Lea, and the collections group > looked into creating a package of big arrays for each of the scaler types. > With such a solution we could do something to allow the array be backed by a > file mapping. This sounds very interesting, but the implementation might be difficult. However, for me it seems that for most use cases the existing NIO buffer API would be fine if it were 64-bit addressable. A BigByteBuffer is just a new API (and developers who need it can use it), but a native support for 64-bit arrays seems to be much more effort (and most developers never need it). So I'd rather vote for big buffers than for big arrays. The ByteBuffer implemenation of Sun's JDK uses the sun.mis.Unsafe class which already seems to be able to address 64 bits (look at ByteBuffer.allocateDirect() and Unsafe.allocateMemory()). So I got the impression that (at least for this implemenation) just a few method signatures would have to be changed (e.g. allocateDirect(long)). -Juergen From jason.rutherglen at gmail.com Thu Jun 11 16:09:19 2009 From: jason.rutherglen at gmail.com (Jason Rutherglen) Date: Thu, 11 Jun 2009 16:09:19 -0700 Subject: madvise(ptr, len, MADV_SEQUENTIAL) Message-ID: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> Is there going to be a way to do this in the new Java IO APIs? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090611/4aa531fc/attachment.html From Alan.Bateman at Sun.COM Mon Jun 15 00:36:21 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Mon, 15 Jun 2009 08:36:21 +0100 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> Message-ID: <4A35F9F5.9070403@sun.com> Jason Rutherglen wrote: > Is there going to be a way to do this in the new Java IO APIs? Good question, as it has come up a few times and is needed for some important use-cases. A while back I looked into adding a MappedByteBuffer#advise method to allow the application provide hints on the expected usage but didn't complete it. We should probably look at this again for jdk7. -Alan. From jason.rutherglen at gmail.com Mon Jun 15 15:01:56 2009 From: jason.rutherglen at gmail.com (Jason Rutherglen) Date: Mon, 15 Jun 2009 15:01:56 -0700 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <4A35F9F5.9070403@sun.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> <4A35F9F5.9070403@sun.com> Message-ID: <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> Thanks Alan. I cross posted this to the Lucene dev list where we are discussing using madvise for minimizing unnecessary IO cache usage when merging segments (where we really want the newly merged segments in the IO cache rather than the old segment files). How would the advise method work? Would there need to be a hint in the FileChannel.map method? -J On Mon, Jun 15, 2009 at 12:36 AM, Alan Bateman wrote: > Jason Rutherglen wrote: > >> Is there going to be a way to do this in the new Java IO APIs? >> > Good question, as it has come up a few times and is needed for some > important use-cases. A while back I looked into adding a > MappedByteBuffer#advise method to allow the application provide hints on the > expected usage but didn't complete it. We should probably look at this again > for jdk7. > > -Alan. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090615/59a1025c/attachment.html From Alan.Bateman at Sun.COM Tue Jun 16 05:53:55 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 16 Jun 2009 13:53:55 +0100 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> <4A35F9F5.9070403@sun.com> <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> Message-ID: <4A3795E3.5010206@sun.com> Jason Rutherglen wrote: > Thanks Alan. > > I cross posted this to the Lucene dev list where we are discussing > using madvise for minimizing unnecessary IO cache usage when merging > segments (where we really want the newly merged segments in the IO > cache rather than the old segment files). > > How would the advise method work? Would there need to be a hint in > the FileChannel.map method? > > -J If we add it then it would probably be a new method on MappedByteBuffer that takes a range and an enum constant (that is the usage hint) as parameters. This would be slightly more flexible than specifying a hint for the entire region when mapping. Also, it would be awkward to retrofit MapMode to include hints because the parameter is a constant rather than a set of modes. -Alan. From lucene at mikemccandless.com Tue Jun 16 07:21:56 2009 From: lucene at mikemccandless.com (Michael McCandless) Date: Tue, 16 Jun 2009 10:21:56 -0400 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> <4A35F9F5.9070403@sun.com> <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> Message-ID: <9ac0c6aa0906160721l4acfcd31t8cfd10529d5384fb@mail.gmail.com> Lucene could really make use of this method. When a segment merge takes place, we can read & write many GB of data, which without madvise on many OSs would effectively flush the IO cache (thus hurting our search performance). Mike On Mon, Jun 15, 2009 at 6:01 PM, Jason Rutherglen wrote: > Thanks Alan. > > I cross posted this to the Lucene dev list where we are discussing using > madvise for minimizing unnecessary IO cache usage when merging segments > (where we really want the newly merged segments in the IO cache rather than > the old segment files). > > How would the advise method work?? Would there need to be a hint in the > FileChannel.map method? > > -J > > On Mon, Jun 15, 2009 at 12:36 AM, Alan Bateman wrote: >> >> Jason Rutherglen wrote: >>> >>> Is there going to be a way to do this in the new Java IO APIs? >> >> Good question, as it has come up a few times and is needed for some >> important use-cases. A while back I looked into adding a >> MappedByteBuffer#advise method to allow the application provide hints on the >> expected usage but didn't complete it. We should probably look at this again >> for jdk7. >> >> -Alan. >> > > From jason.rutherglen at gmail.com Tue Jun 16 20:23:04 2009 From: jason.rutherglen at gmail.com (Jason Rutherglen) Date: Tue, 16 Jun 2009 20:23:04 -0700 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <9ac0c6aa0906160721l4acfcd31t8cfd10529d5384fb@mail.gmail.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> <4A35F9F5.9070403@sun.com> <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> <9ac0c6aa0906160721l4acfcd31t8cfd10529d5384fb@mail.gmail.com> Message-ID: <85d3c3b60906162023y4dd90008y7607028f8a67c101@mail.gmail.com> Alan, Do you think something like FileDescriptor.setAdvise (mirroring posix_fadvise) makes sense? -J On Tue, Jun 16, 2009 at 7:21 AM, Michael McCandless < lucene at mikemccandless.com> wrote: > Lucene could really make use of this method. When a segment merge > takes place, we can read & write many GB of data, which without > madvise on many OSs would effectively flush the IO cache (thus hurting > our search performance). > > Mike > > On Mon, Jun 15, 2009 at 6:01 PM, Jason > Rutherglen wrote: > > Thanks Alan. > > > > I cross posted this to the Lucene dev list where we are discussing using > > madvise for minimizing unnecessary IO cache usage when merging segments > > (where we really want the newly merged segments in the IO cache rather > than > > the old segment files). > > > > How would the advise method work? Would there need to be a hint in the > > FileChannel.map method? > > > > -J > > > > On Mon, Jun 15, 2009 at 12:36 AM, Alan Bateman > wrote: > >> > >> Jason Rutherglen wrote: > >>> > >>> Is there going to be a way to do this in the new Java IO APIs? > >> > >> Good question, as it has come up a few times and is needed for some > >> important use-cases. A while back I looked into adding a > >> MappedByteBuffer#advise method to allow the application provide hints on > the > >> expected usage but didn't complete it. We should probably look at this > again > >> for jdk7. > >> > >> -Alan. > >> > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscribe at lucene.apache.org > For additional commands, e-mail: java-dev-help at lucene.apache.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090616/280950c8/attachment.html From Alan.Bateman at Sun.COM Wed Jun 17 02:19:56 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 17 Jun 2009 10:19:56 +0100 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <85d3c3b60906162023y4dd90008y7607028f8a67c101@mail.gmail.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> <4A35F9F5.9070403@sun.com> <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> <9ac0c6aa0906160721l4acfcd31t8cfd10529d5384fb@mail.gmail.com> <85d3c3b60906162023y4dd90008y7607028f8a67c101@mail.gmail.com> Message-ID: <4A38B53C.9020002@sun.com> Jason Rutherglen wrote: > Alan, > > Do you think something like FileDescriptor.setAdvise (mirroring > posix_fadvise) makes sense? > > -J Something like a posix_fadvise would be more appropriate for FileChannel or maybe as a usage hint when opening the file (the new APIs for opening files are extensible to allow for additional options in the future or even implementation specific options). I don't think we've had much interest in doing this, maybe because it would be a no-op on many operating systems. -Alan. From cowwoc at bbs.darktech.org Wed Jun 17 09:19:30 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Wed, 17 Jun 2009 09:19:30 -0700 (PDT) Subject: Timeout values Message-ID: <1245255570528-3093833.post@n2.nabble.com> Hi, Alan and I discussed NIO2 timeout values a few weeks ago. I wanted to continue this discussion in a more public forum. Here is what was said so far: Gili wrote: > On a separate note, I believe there is inconsistency between > AsynchronousSocketChannel's definition > of timeouts (where timeout==0 means "no timeout") and > http://java.sun.com/javase/6/docs/api/java/util/concurrent/package-summary.html > that states "All methods that accept timeout parameters treat values less > than or equal to zero to > mean not to wait at all. To wait "forever", you can use a value of > Long.MAX_VALUE." > > nio2 "timeout == 0" is equivalent to concurrent "timeout == > Integer.MAXIMUM" and > concurrent "timeout == 0" has no equivalent in nio2. Although nio2's > definition feels more familiar > coming from a C/C++ background, I personally prefer the concurrent > definition. Alan wrote: > You are right that the concurrent and channels package aren't consistent > here but I don't see that we > have an inconsistency in the channels package. > > In the concurrent package then a timeout of <=0 means do not wait. In the > channels package then 0 > means wait forever and <0 is invalid (when I say the channels package I > means Selector#select as > that is the only blocking method with timeout in that package). This > inconsistency has been there for > several releases and hasn't been a major problem (to my knowledge anyway). > > With the asynchronous I/O API the operations are non-blocking so there is > no equivalent to "do not > wait" in this API. Instead, the I/O operation has an associated timer or > it doesn't. That plus we have > tried to be consistent with Selector we have timeout == 0 means no timer > and <0 is invalid. Clearly we > are open to better ways to doing this. For example, one could argument > that <= 0 should mean no > timer but that would be inconsistent with Selector. One could argue that > Long.MAX_VALUE should > mean infinitely but that just makes it more awkward to use when you don't > want a timeout. Wanting to maintain backwards compatibility with Selector is perfectly valid, and I fully appreciate the fact that the asynchronous I/O APIs probably have no need for "do not wait" but I personally believe that it is very important for timeouts usage to be consistent across all the Core APIs. Users should be able to transition from one API to another without having to read the fine print about how timeout works in this package or another. As to the argument that using Long.MAX_VALUE is more awkward than zero, I think it's subjective. On an equally subjective level, I personally find "0 means do not wait, Long.MAX_VALUE means wait forever" a lot more intuitive than the Selector behavior. I'd love to hear what others on this forum think on the matter. Thanks, Gili -- View this message in context: http://n2.nabble.com/Timeout-values-tp3093833p3093833.html Sent from the nio-discuss mailing list archive at Nabble.com. From jason.rutherglen at gmail.com Thu Jun 18 15:32:58 2009 From: jason.rutherglen at gmail.com (Jason Rutherglen) Date: Thu, 18 Jun 2009 15:32:58 -0700 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <4A38B53C.9020002@sun.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> <4A35F9F5.9070403@sun.com> <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> <9ac0c6aa0906160721l4acfcd31t8cfd10529d5384fb@mail.gmail.com> <85d3c3b60906162023y4dd90008y7607028f8a67c101@mail.gmail.com> <4A38B53C.9020002@sun.com> Message-ID: <85d3c3b60906181532q45549febn2dadcc1e9bd0ede9@mail.gmail.com> Hmm... So the list at the bottom of this page looks accurate? http://www.gnu.org/software/hello/manual/gnulib/posix_005ffadvise.html If it is, then posix_fadvise works on Linux only? Perhaps madvise will be better then (judging by the much smaller unsupported list)? It seems to run on most platforms: http://www.gnu.org/software/hello/manual/gnulib/madvise.html On Wed, Jun 17, 2009 at 2:19 AM, Alan Bateman wrote: > Jason Rutherglen wrote: > >> Alan, >> >> Do you think something like FileDescriptor.setAdvise (mirroring >> posix_fadvise) makes sense? >> >> -J >> > Something like a posix_fadvise would be more appropriate for FileChannel or > maybe as a usage hint when opening the file (the new APIs for opening files > are extensible to allow for additional options in the future or even > implementation specific options). I don't think we've had much interest in > doing this, maybe because it would be a no-op on many operating systems. > > -Alan. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090618/ef26bd71/attachment.html From daleelectric at inbox.com Thu Jun 18 22:13:37 2009 From: daleelectric at inbox.com (Dale Electric Supply Co) Date: Fri, 19 Jun 2009 01:13:37 -0400 Subject: Job Offer: Online Representatives Needed Message-ID: <200906190513.n5J5DFmo021729@nlpi117.prodigy.net> An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090619/9b125668/attachment.html From uwe at thetaphi.de Fri Jun 19 01:15:31 2009 From: uwe at thetaphi.de (Uwe Schindler) Date: Fri, 19 Jun 2009 10:15:31 +0200 Subject: madvise(ptr, len, MADV_SEQUENTIAL) In-Reply-To: <85d3c3b60906181532q45549febn2dadcc1e9bd0ede9@mail.gmail.com> References: <85d3c3b60906111609g69156159u447d1e9342f57a97@mail.gmail.com> <4A35F9F5.9070403@sun.com> <85d3c3b60906151501s66c6fffw7fde47042be269e9@mail.gmail.com> <9ac0c6aa0906160721l4acfcd31t8cfd10529d5384fb@mail.gmail.com> <85d3c3b60906162023y4dd90008y7607028f8a67c101@mail.gmail.com> <4A38B53C.9020002@sun.com> <85d3c3b60906181532q45549febn2dadcc1e9bd0ede9@mail.gmail.com> Message-ID: But then we also need to map, when writing to files, which is hard to do, because you do not know how large the mapping area will be (unknown filesize). As Earwin suggested, we could change MMapDirectory to also mmap on writing, but maps the buffers in something we call "pages". Filesizes of written files are then always multiples of this "page size", if this is not a problem for lucene (e.g. making the random access file larger than actually needed, having 0-bytes at the unused end). Maybe we need than an api to tell the directory implementation, how large a segment may become before writing (when merging, this should be possible, as the filesize is about the sum of all merged segments). On windows, we then will also rely on the cleaner()-hack to correctly close files (see current MMap in lucene-trunk, where the cleaner-hack can optionally be enabled, without guaranties). ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: uwe at thetaphi.de _____ From: Jason Rutherglen [mailto:jason.rutherglen at gmail.com] Sent: Friday, June 19, 2009 12:33 AM To: Alan Bateman Cc: java-dev at lucene.apache.org; nio-discuss at openjdk.java.net Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL) Hmm... So the list at the bottom of this page looks accurate? http://www.gnu.org/software/hello/manual/gnulib/posix_005ffadvise.html If it is, then posix_fadvise works on Linux only? Perhaps madvise will be better then (judging by the much smaller unsupported list)? It seems to run on most platforms: http://www.gnu.org/software/hello/manual/gnulib/madvise.html On Wed, Jun 17, 2009 at 2:19 AM, Alan Bateman wrote: Jason Rutherglen wrote: Alan, Do you think something like FileDescriptor.setAdvise (mirroring posix_fadvise) makes sense? -J Something like a posix_fadvise would be more appropriate for FileChannel or maybe as a usage hint when opening the file (the new APIs for opening files are extensible to allow for additional options in the future or even implementation specific options). I don't think we've had much interest in doing this, maybe because it would be a no-op on many operating systems. -Alan. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090619/293a8774/attachment.html From cowwoc at bbs.darktech.org Wed Jun 24 16:27:30 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Wed, 24 Jun 2009 16:27:30 -0700 (PDT) Subject: AsynchronousChannel.close() Message-ID: <1245886050430-3152177.post@n2.nabble.com> Hi, The Javadoc for AsynchronousChannel reads "Where the cancel method is invoked with the mayInterruptIfRunning parameter set to true then the I/O operation may be interrupted by closing the channel. In that case all threads waiting on the result of the I/O operation throw CancellationException and any other I/O operations outstanding on the channel complete with the exception AsynchronousCloseException." I don't understand the last part. Are you implying that all Futures get CancellationException whereas all CompletionHandlers get AsynchronousCloseException? Or did you mean something else? The paragraph makes it seem as if Futures that are not actively being waited on will throw AsynchronousCloseException when get() is eventually invoked. This sounds rather difficult from an implementation point of view. Please clarify. Thanks, Gili -- View this message in context: http://n2.nabble.com/AsynchronousChannel.close%28%29-tp3152177p3152177.html Sent from the nio-discuss mailing list archive at Nabble.com. From cowwoc at bbs.darktech.org Wed Jun 24 17:41:15 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Wed, 24 Jun 2009 17:41:15 -0700 (PDT) Subject: Does AsynchronousSocketChannel.read(ByteBuffer) throw ShutdownChannelGroupException? Message-ID: <1245890475473-3152470.post@n2.nabble.com> AsynchronousSocketChannel.read(ByteBuffer) overrides AsynchronousByteChannel.read(ByteBuffer). The latter has no concept of ChannelGroups so it does not declare that the method may throw ShutdownChannelGroupException. AsynchronousSocketChannel, on the other hand, declares that all I/O operation will throw ShutdownChannelGroupException if the channel group is shut down in the middle of an operation. There seems to be a conflict of specifications. Doesn't Design by Contract specify that a subclass may throw less exceptions but it may not throw new exceptions? I know that technically speaking all methods throw RuntimeException but that's a little dirty. This exception does not indicate programming error and it is quite reasonable to expect that developers will want to catch it. At the very least, AsynchronousSocketChannel.read(ByteBuffer)'s documentation should address this more explicitly. Thank you, Gili -- View this message in context: http://n2.nabble.com/Does-AsynchronousSocketChannel.read%28ByteBuffer%29-throw-ShutdownChannelGroupException--tp3152470p3152470.html Sent from the nio-discuss mailing list archive at Nabble.com. From Alan.Bateman at Sun.COM Thu Jun 25 00:48:52 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 25 Jun 2009 08:48:52 +0100 Subject: AsynchronousChannel.close() In-Reply-To: <1245886050430-3152177.post@n2.nabble.com> References: <1245886050430-3152177.post@n2.nabble.com> Message-ID: <4A432BE4.900@sun.com> Gili wrote: > Hi, > > The Javadoc for AsynchronousChannel reads "Where the cancel method is > invoked with the mayInterruptIfRunning parameter set to true then the I/O > operation may be interrupted by closing the channel. In that case all > threads waiting on the result of the I/O operation throw > CancellationException and any other I/O operations outstanding on the > channel complete with the exception AsynchronousCloseException." > > I don't understand the last part. Are you implying that all Futures get > CancellationException whereas all CompletionHandlers get > AsynchronousCloseException? Or did you mean something else? > > The paragraph makes it seem as if Futures that are not actively being waited > on will throw AsynchronousCloseException when get() is eventually invoked. > This sounds rather difficult from an implementation point of view. Please > clarify. > The spec allows a forceful cancel (mayInterruptIfRunning parameter set to true) to be implemented as an asynchronous close. This will cause "other" I/O operations outstanding on the channel to fail with AsynchronousCloseException. So for example, suppose you forcefully cancel a write while there is read outstanding. All threads waiting on the result of the write will throw CancellationException. Assuming the implementation closes the channel, then it will cause the read to fail with an AsynchronousCloseException. If the read was initiated specifying a completion handler, then the completion handler's failed method will be invoked with the exception (AsynchronousCloseException). Alternatively, if the read was initiated returning a Future then any threads waiting on the read result will throw ExecutionException with AsynchronousCloseException as the cause. Does that help? -Alan. From Alan.Bateman at Sun.COM Thu Jun 25 02:43:15 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 25 Jun 2009 10:43:15 +0100 Subject: Does AsynchronousSocketChannel.read(ByteBuffer) throw ShutdownChannelGroupException? In-Reply-To: <1245890475473-3152470.post@n2.nabble.com> References: <1245890475473-3152470.post@n2.nabble.com> Message-ID: <4A4346B3.3030308@sun.com> Gili wrote: > AsynchronousSocketChannel.read(ByteBuffer) overrides > AsynchronousByteChannel.read(ByteBuffer). The latter has no concept of > ChannelGroups so it does not declare that the method may throw > ShutdownChannelGroupException. AsynchronousSocketChannel, on the other hand, > declares that all I/O operation will throw ShutdownChannelGroupException if > the channel group is shut down in the middle of an operation. > > There seems to be a conflict of specifications. Doesn't Design by Contract > specify that a subclass may throw less exceptions but it may not throw new > exceptions? I know that technically speaking all methods throw > RuntimeException but that's a little dirty. This exception does not indicate > programming error and it is quite reasonable to expect that developers will > want to catch it. At the very least, > AsynchronousSocketChannel.read(ByteBuffer)'s documentation should address > this more explicitly. > It's just an oversight that the ShutdownChannelGroupException is missing from the list of possible exceptions - I'll fix that thanks! As regards shutdown while there are I/O operations outstanding - it depends on if you use the shutdown or shutdownNow methods. The shutdown method does a graceful shutdown and does not impact the outstanding I/O operations. The shutdowNow method is to do a forceful shutdown and this is specified to close all open channels in the group. This causes all outstanding I/O operations to fail with AsynchronousCloseException. -Alan. From cowwoc at bbs.darktech.org Thu Jun 25 07:15:33 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 25 Jun 2009 07:15:33 -0700 (PDT) Subject: AsynchronousChannel.close() In-Reply-To: <4A432BE4.900@sun.com> References: <1245886050430-3152177.post@n2.nabble.com> <4A432BE4.900@sun.com> Message-ID: <1245939333467-3155293.post@n2.nabble.com> Okay, this is what I thought at first. What threw me off is... how can multiple threads be waiting on the result of the same I/O operation? Thanks, Gili Alan Bateman wrote: > > Gili wrote: >> Hi, >> >> The Javadoc for AsynchronousChannel reads "Where the cancel method is >> invoked with the mayInterruptIfRunning parameter set to true then the I/O >> operation may be interrupted by closing the channel. In that case all >> threads waiting on the result of the I/O operation throw >> CancellationException and any other I/O operations outstanding on the >> channel complete with the exception AsynchronousCloseException." >> >> I don't understand the last part. Are you implying that all Futures get >> CancellationException whereas all CompletionHandlers get >> AsynchronousCloseException? Or did you mean something else? >> >> The paragraph makes it seem as if Futures that are not actively being >> waited >> on will throw AsynchronousCloseException when get() is eventually >> invoked. >> This sounds rather difficult from an implementation point of view. Please >> clarify. >> > The spec allows a forceful cancel (mayInterruptIfRunning parameter set > to true) to be implemented as an asynchronous close. This will cause > "other" I/O operations outstanding on the channel to fail with > AsynchronousCloseException. So for example, suppose you forcefully > cancel a write while there is read outstanding. All threads waiting on > the result of the write will throw CancellationException. Assuming the > implementation closes the channel, then it will cause the read to fail > with an AsynchronousCloseException. If the read was initiated specifying > a completion handler, then the completion handler's failed method will > be invoked with the exception (AsynchronousCloseException). > Alternatively, if the read was initiated returning a Future then any > threads waiting on the read result will throw ExecutionException with > AsynchronousCloseException as the cause. Does that help? > > -Alan. > > -- View this message in context: http://n2.nabble.com/AsynchronousChannel.close%28%29-tp3152177p3155293.html Sent from the nio-discuss mailing list archive at Nabble.com. From cowwoc at bbs.darktech.org Thu Jun 25 07:18:13 2009 From: cowwoc at bbs.darktech.org (cowwoc) Date: Thu, 25 Jun 2009 10:18:13 -0400 Subject: Does AsynchronousSocketChannel.read(ByteBuffer) throw ShutdownChannelGroupException? In-Reply-To: <4A4346B3.3030308@sun.com> References: <1245890475473-3152470.post@n2.nabble.com> <4A4346B3.3030308@sun.com> Message-ID: <4A438725.5090902@bbs.darktech.org> Alan Bateman wrote: > Gili wrote: >> AsynchronousSocketChannel.read(ByteBuffer) overrides >> AsynchronousByteChannel.read(ByteBuffer). The latter has no concept of >> ChannelGroups so it does not declare that the method may throw >> ShutdownChannelGroupException. AsynchronousSocketChannel, on the other >> hand, >> declares that all I/O operation will throw >> ShutdownChannelGroupException if >> the channel group is shut down in the middle of an operation. >> >> There seems to be a conflict of specifications. Doesn't Design by >> Contract >> specify that a subclass may throw less exceptions but it may not throw >> new >> exceptions? I know that technically speaking all methods throw >> RuntimeException but that's a little dirty. This exception does not >> indicate >> programming error and it is quite reasonable to expect that developers >> will >> want to catch it. At the very least, >> AsynchronousSocketChannel.read(ByteBuffer)'s documentation should address >> this more explicitly. >> > It's just an oversight that the ShutdownChannelGroupException is missing > from the list of possible exceptions - I'll fix that thanks! > > As regards shutdown while there are I/O operations outstanding - it > depends on if you use the shutdown or shutdownNow methods. The shutdown > method does a graceful shutdown and does not impact the outstanding I/O > operations. The shutdowNow method is to do a forceful shutdown and this > is specified to close all open channels in the group. This causes all > outstanding I/O operations to fail with AsynchronousCloseException. I'm confused. If shutdownNow() causes outstanding operations to throw AsynchronousCloseException when does ShutdownChannelGroupException get thrown? Thanks, Gili From Alan.Bateman at Sun.COM Thu Jun 25 07:25:46 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 25 Jun 2009 15:25:46 +0100 Subject: AsynchronousChannel.close() In-Reply-To: <1245939333467-3155293.post@n2.nabble.com> References: <1245886050430-3152177.post@n2.nabble.com> <4A432BE4.900@sun.com> <1245939333467-3155293.post@n2.nabble.com> Message-ID: <4A4388EA.5080207@sun.com> Gili wrote: > Okay, this is what I thought at first. What threw me off is... how can > multiple threads be waiting on the result of the same I/O operation? > It would be unusual, but in theory you could have several threads blocked on the Future's get method waiting for the result. If you call the cancel method then they will all throw CancellationException. -Alan. From Alan.Bateman at Sun.COM Thu Jun 25 07:34:05 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 25 Jun 2009 15:34:05 +0100 Subject: Does AsynchronousSocketChannel.read(ByteBuffer) throw ShutdownChannelGroupException? In-Reply-To: <4A438725.5090902@bbs.darktech.org> References: <1245890475473-3152470.post@n2.nabble.com> <4A4346B3.3030308@sun.com> <4A438725.5090902@bbs.darktech.org> Message-ID: <4A438ADD.1060404@sun.com> cowwoc wrote: > : > I'm confused. If shutdownNow() causes outstanding operations to > throw AsynchronousCloseException when does > ShutdownChannelGroupException get thrown? If you look at the open methods you will see that they are specified to throw ShutdownChannelGroupException if you attempt to open a channel in a group that is shutdown. Also, the I/O methods where you specify a completion handler, and attempt to initiate an I/O operation on a channel in a group that has terminated. This case has to fail with the initiating thread throwing an exception (because the completion handler cannot be invoked). -Alan. From cowwoc at bbs.darktech.org Thu Jun 25 08:35:31 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 25 Jun 2009 08:35:31 -0700 (PDT) Subject: Why doesn't CharBuffer.put(String) take a CharSequence? Message-ID: <1245944131845-3155780.post@n2.nabble.com> Hi, I'm curious, why does http://java.sun.com/javase/6/docs/api/java/nio/CharBuffer.html#put(java.lang.String) take a String instead of a CharSequence? Thank you, Gili -- View this message in context: http://n2.nabble.com/Why-doesn%27t-CharBuffer.put%28String%29-take-a-CharSequence--tp3155780p3155780.html Sent from the nio-discuss mailing list archive at Nabble.com. From cowwoc at bbs.darktech.org Thu Jun 25 09:56:22 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 25 Jun 2009 09:56:22 -0700 (PDT) Subject: AsynchronousCharChannel Message-ID: <1245948982400-3156277.post@n2.nabble.com> Hi, I just wanted to let you know I'm almost done implementing a AsynchronousCharChannel that takes a AsynchronousByteChannel as input and provides the following functionality: read/write CharBuffer readLine() I hope no one else is working on the same thing... When I'm done, is there a chance this will end up in the official NIO2 release? Gili -- View this message in context: http://n2.nabble.com/AsynchronousCharChannel-tp3156277p3156277.html Sent from the nio-discuss mailing list archive at Nabble.com. From cowwoc at bbs.darktech.org Thu Jun 25 10:06:21 2009 From: cowwoc at bbs.darktech.org (cowwoc) Date: Thu, 25 Jun 2009 13:06:21 -0400 Subject: Does AsynchronousSocketChannel.read(ByteBuffer) throw ShutdownChannelGroupException? In-Reply-To: <4A438ADD.1060404@sun.com> References: <1245890475473-3152470.post@n2.nabble.com> <4A4346B3.3030308@sun.com> <4A438725.5090902@bbs.darktech.org> <4A438ADD.1060404@sun.com> Message-ID: <4A43AE8D.8080308@bbs.darktech.org> Alan Bateman wrote: > cowwoc wrote: >> : >> I'm confused. If shutdownNow() causes outstanding operations to >> throw AsynchronousCloseException when does >> ShutdownChannelGroupException get thrown? > If you look at the open methods you will see that they are specified to > throw ShutdownChannelGroupException if you attempt to open a channel in > a group that is shutdown. > > Also, the I/O methods where you specify a completion handler, and > attempt to initiate an I/O operation on a channel in a group that has > terminated. This case has to fail with the initiating thread throwing an > exception (because the completion handler cannot be invoked). > > -Alan. Last I checked, Future does not guarantee that its implementation is thread safe. Does it? Gili From studdugie at gmail.com Thu Jun 25 11:18:42 2009 From: studdugie at gmail.com (Dane Foster) Date: Thu, 25 Jun 2009 14:18:42 -0400 Subject: Why doesn't CharBuffer.put(String) take a CharSequence? In-Reply-To: <1245944131845-3155780.post@n2.nabble.com> References: <1245944131845-3155780.post@n2.nabble.com> Message-ID: <5a59ce530906251118m3d9a7bcbm249a8834958cd53b@mail.gmail.com> Because String IS A CharSequence. On Thu, Jun 25, 2009 at 11:35 AM, Gili wrote: > > Hi, > > I'm curious, why does > http://java.sun.com/javase/6/docs/api/java/nio/CharBuffer.html#put(java.lang.String) > take a String instead of a CharSequence? > > Thank you, > Gili > -- > View this message in context: http://n2.nabble.com/Why-doesn%27t-CharBuffer.put%28String%29-take-a-CharSequence--tp3155780p3155780.html > Sent from the nio-discuss mailing list archive at Nabble.com. > > From Alan.Bateman at Sun.COM Thu Jun 25 12:41:38 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 25 Jun 2009 20:41:38 +0100 Subject: AsynchronousCharChannel In-Reply-To: <1245948982400-3156277.post@n2.nabble.com> References: <1245948982400-3156277.post@n2.nabble.com> Message-ID: <4A43D2F2.1090308@sun.com> Gili wrote: > Hi, > > I just wanted to let you know I'm almost done implementing a > AsynchronousCharChannel that takes a AsynchronousByteChannel as input and > provides the following functionality: > > read/write CharBuffer > readLine() > > I hope no one else is working on the same thing... When I'm done, is there a > chance this will end up in the official NIO2 release? > > Gili > Alex Libman has built a filter framework on this API and it includes an AsynchronousCoder for char I/O. You might want to bring it up on the nio-dev mailing list for more discussion. -Alan. From Alan.Bateman at Sun.COM Thu Jun 25 12:44:46 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 25 Jun 2009 20:44:46 +0100 Subject: Why doesn't CharBuffer.put(String) take a CharSequence? In-Reply-To: <1245944131845-3155780.post@n2.nabble.com> References: <1245944131845-3155780.post@n2.nabble.com> Message-ID: <4A43D3AE.9000703@sun.com> Gili wrote: > Hi, > > I'm curious, why does > http://java.sun.com/javase/6/docs/api/java/nio/CharBuffer.html#put(java.lang.String) > take a String instead of a CharSequence? > > Thank you, > Gili > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4860681 From cowwoc at bbs.darktech.org Thu Jun 25 12:54:59 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 25 Jun 2009 12:54:59 -0700 (PDT) Subject: Why doesn't CharBuffer.put(String) take a CharSequence? In-Reply-To: <4A43D3AE.9000703@sun.com> References: <1245944131845-3155780.post@n2.nabble.com> <4A43D3AE.9000703@sun.com> Message-ID: <1245959699102-3157343.post@n2.nabble.com> Any chance you can roll this in as part of JDK7? :) Right now my code has to use fragile "instanceof" code. It will break if new implementations are added in the future. I took a look at the underlying methods for put(CharBuffer) and put(String) and they seem to be doing the exact same thing, just using get() instead of chatAt(). Gili Alan Bateman wrote: > > Gili wrote: >> Hi, >> >> I'm curious, why does >> http://java.sun.com/javase/6/docs/api/java/nio/CharBuffer.html#put(java.lang.String) >> take a String instead of a CharSequence? >> >> Thank you, >> Gili >> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4860681 > > -- View this message in context: http://n2.nabble.com/Why-doesn%27t-CharBuffer.put%28String%29-take-a-CharSequence--tp3155780p3157343.html Sent from the nio-discuss mailing list archive at Nabble.com. From i30817 at gmail.com Thu Jun 25 13:08:46 2009 From: i30817 at gmail.com (Paulo Levi) Date: Thu, 25 Jun 2009 21:08:46 +0100 Subject: Why doesn't CharBuffer.put(String) take a CharSequence? In-Reply-To: <1245959699102-3157343.post@n2.nabble.com> References: <1245944131845-3155780.post@n2.nabble.com> <4A43D3AE.9000703@sun.com> <1245959699102-3157343.post@n2.nabble.com> Message-ID: <212322090906251308m3dcdea10t30d80f17a1d5223@mail.gmail.com> Wouldn't it be nice if CharSequence had the getChars method of String? I fear charAt makes algorithms that need buffered copies much slower. I already asked why not, but adding methods is a incompatible change for interfaces. In this special case, if CharSequence was a abstract class, it could be easily be implemented by charAt, and overridden for performance in implementations, much like InputStream is now. In fact much the same thing could be said for sort in collections. Missing abstract methods tend to be able to be implemented by primitive methods. The reason i don't like to use interfaces (except enums that can't extend anything, and are very useful to discoverability). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090625/af4fb0c1/attachment.html From cowwoc at bbs.darktech.org Thu Jun 25 15:02:49 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 25 Jun 2009 15:02:49 -0700 (PDT) Subject: Timeout values In-Reply-To: <1245255570528-3093833.post@n2.nabble.com> References: <1245255570528-3093833.post@n2.nabble.com> Message-ID: <1245967369151-3158082.post@n2.nabble.com> Did this message ever get accepted by the mailing list? I ask because http://n2.nabble.com/Timeout-values-td3093833.html seems to believe otherwise. Thanks, Gili Gili wrote: > > Hi, > > Alan and I discussed NIO2 timeout values a few weeks ago. I wanted to > continue this discussion in a more public forum. Here is what was said so > far: > > Gili wrote: >> On a separate note, I believe there is inconsistency between >> AsynchronousSocketChannel's definition >> of timeouts (where timeout==0 means "no timeout") and >> http://java.sun.com/javase/6/docs/api/java/util/concurrent/package-summary.html >> that states "All methods that accept timeout parameters treat values less >> than or equal to zero to >> mean not to wait at all. To wait "forever", you can use a value of >> Long.MAX_VALUE." >> >> nio2 "timeout == 0" is equivalent to concurrent "timeout == >> Integer.MAXIMUM" and >> concurrent "timeout == 0" has no equivalent in nio2. Although nio2's >> definition feels more familiar >> coming from a C/C++ background, I personally prefer the concurrent >> definition. > > Alan wrote: >> You are right that the concurrent and channels package aren't consistent >> here but I don't see that we >> have an inconsistency in the channels package. >> >> In the concurrent package then a timeout of <=0 means do not wait. In the >> channels package then 0 >> means wait forever and <0 is invalid (when I say the channels package I >> means Selector#select as >> that is the only blocking method with timeout in that package). This >> inconsistency has been there for >> several releases and hasn't been a major problem (to my knowledge >> anyway). >> >> With the asynchronous I/O API the operations are non-blocking so there is >> no equivalent to "do not >> wait" in this API. Instead, the I/O operation has an associated timer or >> it doesn't. That plus we have >> tried to be consistent with Selector we have timeout == 0 means no timer >> and <0 is invalid. Clearly we >> are open to better ways to doing this. For example, one could argument >> that <= 0 should mean no >> timer but that would be inconsistent with Selector. One could argue that >> Long.MAX_VALUE should >> mean infinitely but that just makes it more awkward to use when you don't >> want a timeout. > > Wanting to maintain backwards compatibility with Selector is perfectly > valid, and I fully appreciate the fact that the asynchronous I/O APIs > probably have no need for "do not wait" but I personally believe that it > is very important for timeouts usage to be consistent across all the Core > APIs. Users should be able to transition from one API to another without > having to read the fine print about how timeout works in this package or > another. > > As to the argument that using Long.MAX_VALUE is more awkward than zero, I > think it's subjective. On an equally subjective level, I personally find > "0 means do not wait, Long.MAX_VALUE means wait forever" a lot more > intuitive than the Selector behavior. > > I'd love to hear what others on this forum think on the matter. > > Thanks, > Gili > -- View this message in context: http://n2.nabble.com/Timeout-values-tp3093833p3158082.html Sent from the nio-discuss mailing list archive at Nabble.com. From cowwoc at bbs.darktech.org Thu Jun 25 15:26:08 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Thu, 25 Jun 2009 15:26:08 -0700 (PDT) Subject: AsynchronousCharChannel In-Reply-To: <4A43D2F2.1090308@sun.com> References: <1245948982400-3156277.post@n2.nabble.com> <4A43D2F2.1090308@sun.com> Message-ID: <1245968768635-3158181.post@n2.nabble.com> I found Alex's JProactor framework here: http://jproactor.svn.sourceforge.net/viewvc/jproactor/trunk/nio2/src/au/com/terabit/nio2/filter/ I have two problems with AsynchronousCoder. 1) The code is sparsely documented. 2) It's too "abstract". I'm looking for a practical/simple solution to a common use-case. I don't think it makes sense to implement readLine() the same way as read(CharBuffer) because in the former case you know how many characters the buffer must contain whereas in the latter case you have no idea how long the line will be. That's why BufferedReader.readLine() returns a String instead of taking in a char[]. Anyway, I think we better wait until I finish implementing this API. Once I'm done I will post the code for your review and you can judge how well it fits (or not) alongside NIO2. Thank you, Gili Alan Bateman wrote: > > Gili wrote: >> Hi, >> >> I just wanted to let you know I'm almost done implementing a >> AsynchronousCharChannel that takes a AsynchronousByteChannel as input and >> provides the following functionality: >> >> read/write CharBuffer >> readLine() >> >> I hope no one else is working on the same thing... When I'm done, is >> there a >> chance this will end up in the official NIO2 release? >> >> Gili >> > Alex Libman has built a filter framework on this API and it includes an > AsynchronousCoder for char I/O. You might want to bring it up on the > nio-dev mailing list for more discussion. > > -Alan. > > -- View this message in context: http://n2.nabble.com/AsynchronousCharChannel-tp3156277p3158181.html Sent from the nio-discuss mailing list archive at Nabble.com. From Alan.Bateman at Sun.COM Fri Jun 26 02:50:03 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 26 Jun 2009 10:50:03 +0100 Subject: Timeout values In-Reply-To: <1245967369151-3158082.post@n2.nabble.com> References: <1245255570528-3093833.post@n2.nabble.com> <1245967369151-3158082.post@n2.nabble.com> Message-ID: <4A4499CB.8090505@sun.com> Gili wrote: > Did this message ever get accepted by the mailing list? I ask because > http://n2.nabble.com/Timeout-values-td3093833.html seems to believe > otherwise. > > Thanks, > Gili > Yes, I saw it, but didn't see any replies. Just to re-cap, the methods that wait-with-timeout introduced by this API are: AsynchronousChannelGroup#awaitTermination(long, TimeUnit), and WatchService#poll(long, TimeUnit) The timeouts in both cases are specified to be consistent with j.u.c. That is, a timeout of <= 0 means do not wait. The only pre-existing method in this API that can wait-with-timeout is Selector's select(long) method. A timeout of 0 means wait indefinitely; a negative timeout is not allowed. To do a non-blocking select you use the selectNow() method. You are right that this is a bit inconsistent with j.u.c but I don't think anyone has complained. Personally, I think it's okay because blocking and non-blocking selects are used for different purposes. In any case, Selector was added in 1.4 and we can't change it. Moving onto AsynchronousSocketChannel, which can optionally associate a timer with an I/O operation. Here, the timeout is to cause the I/O operation to fail if the I/O doesn't complete before the timeout expires. The timeout is not a wait timeout so different semantics to the above. There are two sets of methods, depending on if you using a timeout or not. If you don't want a timeout then you will typically using the 3-argument: read(ByteBuffer, Object, CompletionHandler) whereas if you want a timeout then you will use the 5-parameter methods: read(ByteBuffer, long, TimeUnit, Object, CompletionHandler) Your concern is with 5-parameter methods for cases where you don't want a timer. This usage will likely be rare (except for the scatter/gather case which don't have overloading for all cases). As specified now, you pass a timeout of 0L to mean "no timeout". A negative timeout is not allowed (that was for consistency with Selector; another choice would be for <= 0 to mean no timeout and so IllegalArgumentException is not thrown if you pass a negative value).. If I recall correctly, you are suggesting that Long.MAX_VALUE would be a better choice. That is, change the wording slightly so that MAX_VALUE means an infinite timeout (so that the implementation doesn't need to start a timer). I don't recall if you had a suggestion for <= 0 but in any case, I don't see a big motive to change this. There's nothing stopping you passing in MAX_VALUE now except that it's slightly less efficient than 0. -Alan. From dl at cs.oswego.edu Fri Jun 26 05:40:55 2009 From: dl at cs.oswego.edu (Doug Lea) Date: Fri, 26 Jun 2009 08:40:55 -0400 Subject: Timeout values In-Reply-To: <4A4499CB.8090505@sun.com> References: <1245255570528-3093833.post@n2.nabble.com> <1245967369151-3158082.post@n2.nabble.com> <4A4499CB.8090505@sun.com> Message-ID: <4A44C1D7.6070600@cs.oswego.edu> > > Just to re-cap, the methods that wait-with-timeout introduced by this > API are: > AsynchronousChannelGroup#awaitTermination(long, TimeUnit), and > WatchService#poll(long, TimeUnit) > > The timeouts in both cases are specified to be consistent with j.u.c. > That is, a timeout of <= 0 means do not wait. BTW, besides conforming to common sense, the main reason for this is to make waits composable. As in: void composite(timeLimit) { ... step1(timeLimit); ... step2(timeLimit - elapsed); } which otherwise can surprisingly misbehave if timeLimit - elapsed happens to be 0. > > The only pre-existing method in this API that can wait-with-timeout is > Selector's select(long) method. A timeout of 0 means wait indefinitely; > a negative timeout is not allowed. To do a non-blocking select you use > the selectNow() method. You are right that this is a bit inconsistent > with j.u.c but I don't think anyone has complained. Personally, I think > it's okay because blocking and non-blocking selects are used for > different purposes. In any case, Selector was added in 1.4 and we can't > change it. It was done this way in nio because the above arguments were not enough to convince those people accustomed to C select/poll APIs, especially since TimeUnit had not yet been introduced, and without it there was no good alternative convention to point to. > > whereas if you want a timeout then you will use the 5-parameter methods: > read(ByteBuffer, long, TimeUnit, Object, CompletionHandler) > > Your concern is with 5-parameter methods for cases where you don't want > a timer. This usage will likely be rare (except for the scatter/gather > case which don't have overloading for all cases). As specified now, you > pass a timeout of 0L to mean "no timeout". I agree that this should be changed. Any method taking a TimeUnit should obey j.u.c conventions. Which means separating out the no-timeout case as another method. Which would be a minor nuisance to carry out but will probably result in fewer usage errors. -Doug From Alan.Bateman at Sun.COM Fri Jun 26 14:14:45 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Fri, 26 Jun 2009 22:14:45 +0100 Subject: Timeout values In-Reply-To: <4A44C1D7.6070600@cs.oswego.edu> References: <1245255570528-3093833.post@n2.nabble.com> <1245967369151-3158082.post@n2.nabble.com> <4A4499CB.8090505@sun.com> <4A44C1D7.6070600@cs.oswego.edu> Message-ID: <4A453A45.30808@sun.com> Doug Lea wrote: > : > BTW, besides conforming to common sense, the main reason for this is > to make waits composable. As in: > void composite(timeLimit) { > ... > step1(timeLimit); > ... > step2(timeLimit - elapsed); > } > which otherwise can surprisingly misbehave if timeLimit - elapsed > happens to be 0. That is a useful property - thanks! > : > It was done this way in nio because the above arguments were > not enough to convince those people accustomed to C select/poll > APIs, especially since TimeUnit had not yet been introduced, > and without it there was no good alternative convention to point to. Although I don't think it carried over completely in that a negative value to poll usually means an indefinite wait. In any case, we need to watch for further APIs bringing timeouts to make sure that inconsistencies don't creep in. It's a pity we can't do anything with the existing APIs. > >> >> whereas if you want a timeout then you will use the 5-parameter methods: >> read(ByteBuffer, long, TimeUnit, Object, CompletionHandler) >> >> Your concern is with 5-parameter methods for cases where you don't >> want a timer. This usage will likely be rare (except for the >> scatter/gather case which don't have overloading for all cases). As >> specified now, you pass a timeout of 0L to mean "no timeout". > > I agree that this should be changed. Any method taking a TimeUnit > should obey j.u.c conventions. Which means separating out the > no-timeout case as another method. Which would be a minor > nuisance to carry out but will probably result in fewer > usage errors. I'll fix this shortly so that <= 0 means no-timeout. It's a minor change but if it does result in fewer usage errors, as you suggest, then it's definitely worth it. Thanks for jumping in to make sure we get this right. -Alan. From briandellisanti at gmail.com Sat Jun 27 11:03:59 2009 From: briandellisanti at gmail.com (Brian Dellisanti) Date: Sat, 27 Jun 2009 14:03:59 -0400 Subject: DirectoryStream Iterator Message-ID: <73cf5f440906271103t4c064184t8c767f2f9ae4c263@mail.gmail.com> DirectoryStream docs say: "The iterator's remove method removes the directory entry for the last element returned by the iterator, as if by invoking the delete method. If an I/O error or security exception occurs then ConcurrentModificationException is thrown with the cause." Will the iterator remain valid after the ConcurrentModificationException? From briandellisanti at gmail.com Sat Jun 27 11:07:35 2009 From: briandellisanti at gmail.com (Brian Dellisanti) Date: Sat, 27 Jun 2009 14:07:35 -0400 Subject: java.nio.channels documentation out of date Message-ID: <73cf5f440906271107i49e7fd5di4e2f0c0ac185ac4e@mail.gmail.com> The java.nio.channels package documentation references MappedBigByteBuffer, which no longer exists. From briandellisanti at gmail.com Sat Jun 27 11:12:07 2009 From: briandellisanti at gmail.com (Brian Dellisanti) Date: Sat, 27 Jun 2009 14:12:07 -0400 Subject: File.getCanonicalPath() and Path.toRealPath() Message-ID: <73cf5f440906271112o59a47596gfd6cbef590f9990a@mail.gmail.com> Is there any guaranteed relationship between the result of File.getCanonicalPath() and the result of Path.toRealPath(true) with the default provider? From Alan.Bateman at Sun.COM Sat Jun 27 13:06:16 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 27 Jun 2009 21:06:16 +0100 Subject: java.nio.channels documentation out of date In-Reply-To: <73cf5f440906271107i49e7fd5di4e2f0c0ac185ac4e@mail.gmail.com> References: <73cf5f440906271107i49e7fd5di4e2f0c0ac185ac4e@mail.gmail.com> Message-ID: <4A467BB8.6050900@sun.com> Brian Dellisanti wrote: > The java.nio.channels package documentation references > MappedBigByteBuffer, which no longer exists. > Thanks - we'll fix that. -Alan. From Alan.Bateman at Sun.COM Sat Jun 27 13:09:24 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 27 Jun 2009 21:09:24 +0100 Subject: File.getCanonicalPath() and Path.toRealPath() In-Reply-To: <73cf5f440906271112o59a47596gfd6cbef590f9990a@mail.gmail.com> References: <73cf5f440906271112o59a47596gfd6cbef590f9990a@mail.gmail.com> Message-ID: <4A467C74.5090009@sun.com> Brian Dellisanti wrote: > Is there any guaranteed relationship between the result of > File.getCanonicalPath() and the result of Path.toRealPath(true) with > the default provider? > A good question! If java.io.File f exists, then f.getCanonicalFile().toPath() should be equal to f.toPath().toRealPath(true). I say "should" because it isn't guaranteed and you will find cases where it won't hold. For example File doesn't work with sym links on Windows (yet), or it doesn't preserve the platform representation of the file path. -Alan. From Alan.Bateman at Sun.COM Sat Jun 27 13:10:42 2009 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Sat, 27 Jun 2009 21:10:42 +0100 Subject: DirectoryStream Iterator In-Reply-To: <73cf5f440906271103t4c064184t8c767f2f9ae4c263@mail.gmail.com> References: <73cf5f440906271103t4c064184t8c767f2f9ae4c263@mail.gmail.com> Message-ID: <4A467CC2.3020006@sun.com> Brian Dellisanti wrote: > DirectoryStream docs say: > > "The iterator's remove method removes the directory entry for the last > element returned by the iterator, as if by invoking the delete method. > If an I/O error or security exception occurs then > ConcurrentModificationException is thrown with the cause." > > Will the iterator remain valid after the ConcurrentModificationException? > Yes, failure to delete the file won't impact the iterator. -Alan. From cowwoc at bbs.darktech.org Sun Jun 28 15:13:27 2009 From: cowwoc at bbs.darktech.org (Gili) Date: Sun, 28 Jun 2009 15:13:27 -0700 (PDT) Subject: What to do about write(CharBuffer)? Message-ID: <1246227207539-3171370.post@n2.nabble.com> Hi, As discussed before, I've got a class that reads/writes CharBuffers on top of an existing AsynchronousByteChannel. I'd like your opinion on the following problem. When a user writes out a CharBuffer, I use CharsetEncoder.encode() to convert it to bytes. The value of CharsetEncoder.encode()'s "endOfInput" argument is used to trigger malformed-character errors. It's not clear how I'm supposed to know whether I've reach the end of input... So, does that imply I should change the method signature from write(CharBuffer) to write(CharBuffer, endOfInput)? Thanks, Gili -- View this message in context: http://n2.nabble.com/What-to-do-about-write%28CharBuffer%29--tp3171370p3171370.html Sent from the nio-discuss mailing list archive at Nabble.com. From martinrb at google.com Sun Jun 28 20:33:24 2009 From: martinrb at google.com (Martin Buchholz) Date: Sun, 28 Jun 2009 20:33:24 -0700 Subject: What to do about write(CharBuffer)? In-Reply-To: <1246227207539-3171370.post@n2.nabble.com> References: <1246227207539-3171370.post@n2.nabble.com> Message-ID: <1ccfd1c10906282033w520c5093j177621a2da9cf3f5@mail.gmail.com> In the general case, an encoder needs to know whether there is more input, not just for error detection, but also for producing the correct output. If some encoder produced different output for x and x + UMLAUT then if the last char is 'x' we need some more input or we need to know that there will be no more. Again in the general case the encoder will need to maintain state between encodes of CharBuffers, and the behavior may be different on the final one. Martin On Sun, Jun 28, 2009 at 15:13, Gili wrote: > > Hi, > > As discussed before, I've got a class that reads/writes CharBuffers on top > of an existing AsynchronousByteChannel. I'd like your opinion on the > following problem. > > When a user writes out a CharBuffer, I use CharsetEncoder.encode() to > convert it to bytes. The value of CharsetEncoder.encode()'s "endOfInput" > argument is used to trigger malformed-character errors. It's not clear how > I'm supposed to know whether I've reach the end of input... > > So, does that imply I should change the method signature from > write(CharBuffer) to write(CharBuffer, endOfInput)? > > Thanks, > Gili > -- > View this message in context: > http://n2.nabble.com/What-to-do-about-write%28CharBuffer%29--tp3171370p3171370.html > Sent from the nio-discuss mailing list archive at Nabble.com. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090628/ad5beca1/attachment.html