From JNord at nds.com Tue Dec 16 09:48:10 2008 From: JNord at nds.com (Nord, James) Date: Tue, 16 Dec 2008 17:48:10 -0000 Subject: Thread pooling in AIO/ Message-ID: Hi all, In testing AsyncIO we've noticed if we want to configure the thread pool to use n threads we need to do 2 things. 1) create an Executor service with Executors.newFixedThreadPool(n) 2) AsynchronousChannelGroup.withFixedThreadPool(executor, n) Why do we need to provide n twice? Shouldn't the ASyncChannel group just farm off something to the exectutor service and block if needed? The other thing is the default is awfull in our tests - as it seems to create a new Thread in the pool for each AsyncDatagramChannel (for 100 multicasts received there were 130+ java threads). Also isn't it the job of the Executor to choose chached pooled or non pooled not the AsyncChannelGroup? (the pooled version in our test be resulted in 130+ java threads vs 30 when we "tuned" it to a fixed number of 8) Regards, /James ********************************************************************************************************* This e-mail is confidential, the property of NDS Ltd and intended for the addressee only. Any dissemination, copying or distribution of this message or any attachments by anyone other than the intended recipient is strictly prohibited. If you have received this message in error, please immediately notify the postmaster at nds.com and destroy the original message. Messages sent to and from NDS may be monitored. NDS cannot guarantee any message delivery method is secure or error-free. Information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. We do not accept responsibility for any errors or omissions in this message and/or attachment that arise as a result of transmission. You should carry out your own virus checks before opening any attachment. Any views or opinions presented are solely those of the author and do not necessarily represent those of NDS. To protect the environment please do not print this e-mail unless necessary. NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 ********************************************************************************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20081216/da3aeb4e/attachment.html From david.lloyd at redhat.com Tue Dec 16 10:18:01 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 16 Dec 2008 12:18:01 -0600 Subject: Thread pooling in AIO? In-Reply-To: References: Message-ID: <4947F0D9.1010309@redhat.com> On 12/16/2008 11:48 AM, Nord, James wrote: > The other thing is the default is awfull in our tests - as it seems to > create a new Thread in the pool for each AsyncDatagramChannel (for 100 > multicasts received there were 130+ java threads). That's the way that fixed thread pools work. The core size is equal to the pool size. If ThreadPoolExecutor.execute() is called and the number of threads is below the core size, then a new thread is created rather than reusing an old one. I think a fixed pool is probably the wrong choice for async I/O - it would make more sense to use a cached pool which can scale up on demand, with a sensible timeout for non-core threads. One must also consider the task rejection policy - I don't think it would be a good thing for async I/O tasks to be rejected since the caller would have no idea what happened to the operation. It would just "disappear". As for the other questions, I can't answer. :-) - DML From Alan.Bateman at Sun.COM Tue Dec 16 10:30:32 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 16 Dec 2008 18:30:32 +0000 Subject: Thread pooling in AIO/ In-Reply-To: References: Message-ID: <4947F3C8.3060101@sun.com> Nord, James wrote: > Hi all, > > In testing AsyncIO we've noticed if we want to configure the thread > pool to use n threads we need to do 2 things. > > 1) create an Executor service with Executors.newFixedThreadPool(n) > 2) AsynchronousChannelGroup.withFixedThreadPool(executor, n) > > Why do we need to provide n twice? Shouldn't the ASyncChannel group > just farm off something to the exectutor service and block if needed? Yes, this is annoying and has already been fixed for the next build, so you can withFixedThreadPool(nThreads, threadFactory). > > The other thing is the default is awfull in our tests - as it seems to > create a new Thread in the pool for each AsyncDatagramChannel (for > 100 multicasts received there were 130+ java threads). Is this the default group or your own group? Also, is this 100 outstanding receive operations on one/many channels or 100 receives (one after the other)? One thing to say about AsynchronousDatagramChannel is that it is currently just a simple implementation and isn't plumbed into the event port mechanism yet. This hasn't been a problem so far because the number of AsynchronousDatagramChannel is expected to be small. I assume you don't observed this with the other network channels as they work very differently. > > Also isn't it the job of the Executor to choose chached pooled or non > pooled not the AsyncChannelGroup? The default group uses a cached thread because it may be shared by several applications. For other groups then you supply the thread pool. -Alan. From Alan.Bateman at Sun.COM Tue Dec 16 10:41:43 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Tue, 16 Dec 2008 18:41:43 +0000 Subject: Thread pooling in AIO? In-Reply-To: <4947F0D9.1010309@redhat.com> References: <4947F0D9.1010309@redhat.com> Message-ID: <4947F667.8050901@sun.com> David M. Lloyd wrote: > I think a fixed pool is probably the wrong choice for async I/O - it > would make more sense to use a cached pool which can scale up on > demand, with a sensible timeout for non-core threads. If the server is completely event driven then a fixed thread pool can work very well. One word of warning is one needs to be careful to avoid idle timeout/keep alive on Windows due to the way that I/O works on that platform. I/O operations are tied in the kernel to the initiating thread; if a thread terminates (because it is idle) then outstanding I/O operations that it has initiated may be aborted. -Alan. From david.lloyd at redhat.com Tue Dec 16 16:04:59 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 16 Dec 2008 18:04:59 -0600 Subject: Thread pooling in AIO? In-Reply-To: <4947F667.8050901@sun.com> References: <4947F0D9.1010309@redhat.com> <4947F667.8050901@sun.com> Message-ID: <4948422B.5000909@redhat.com> On 12/16/2008 12:41 PM, Alan Bateman wrote: > David M. Lloyd wrote: >> I think a fixed pool is probably the wrong choice for async I/O - it >> would make more sense to use a cached pool which can scale up on >> demand, with a sensible timeout for non-core threads. > If the server is completely event driven then a fixed thread pool can > work very well. > > One word of warning is one needs to be careful to avoid idle > timeout/keep alive on Windows due to the way that I/O works on that > platform. I/O operations are tied in the kernel to the initiating > thread; if a thread terminates (because it is idle) then outstanding I/O > operations that it has initiated may be aborted. Interesting. To me that suggests that there ought to be a special Executor or ThreadFactory implementation for the Windows case then - one which knows not to kill a thread if there's an outstanding operation on it. - DML From david.lloyd at redhat.com Tue Dec 16 16:15:35 2008 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 16 Dec 2008 18:15:35 -0600 Subject: Thread pooling in AIO? In-Reply-To: <4947F667.8050901@sun.com> References: <4947F0D9.1010309@redhat.com> <4947F667.8050901@sun.com> Message-ID: <494844A7.5090808@redhat.com> On 12/16/2008 12:41 PM, Alan Bateman wrote: > David M. Lloyd wrote: >> I think a fixed pool is probably the wrong choice for async I/O - it >> would make more sense to use a cached pool which can scale up on >> demand, with a sensible timeout for non-core threads. > If the server is completely event driven then a fixed thread pool can > work very well. I should have said, "a LARGE fixed pool is probably the wrong choice"... - DML From Alan.Bateman at Sun.COM Wed Dec 17 00:55:48 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Wed, 17 Dec 2008 08:55:48 +0000 Subject: Thread pooling in AIO? In-Reply-To: <4948422B.5000909@redhat.com> References: <4947F0D9.1010309@redhat.com> <4947F667.8050901@sun.com> <4948422B.5000909@redhat.com> Message-ID: <4948BE94.5000507@sun.com> David M. Lloyd wrote: > : > Interesting. To me that suggests that there ought to be a special > Executor or ThreadFactory implementation for the Windows case then - > one which knows not to kill a thread if there's an outstanding > operation on it. The default thread pool does the right thing but we don't have a solution for the case where the user/application provides the ExecutorService. A possible hack is to specifically handle the ThreadPoolExecutor case and invoke its setKeepAliveTime to disable trimming back of the thread pool. That doesn't solve the issue completely though as it is possible for a completion handler to terminate due to an uncaught error or runtime exception. For that case the thread is allowed to terminate (ensures the uncaught exception handler is invoked, etc.) but it sets us up for potential side effects. It has not been a problem to date and clearly this scenario is the result of a bug somewhere. -Alan. From JNord at nds.com Thu Dec 18 02:19:37 2008 From: JNord at nds.com (Nord, James) Date: Thu, 18 Dec 2008 10:19:37 -0000 Subject: Thread pooling in AIO/ In-Reply-To: <4947F3C8.3060101@sun.com> References: <4947F3C8.3060101@sun.com> Message-ID: Hi Alan, Comments inline. /James > > In testing AsyncIO we've noticed if we want to configure the thread > > pool to use n threads we need to do 2 things. > > > > 1) create an Executor service with Executors.newFixedThreadPool(n) > > 2) AsynchronousChannelGroup.withFixedThreadPool(executor, n) > > > > Why do we need to provide n twice? Shouldn't the ASyncChannel group > > just farm off something to the exectutor service and block if needed? > Yes, this is annoying and has already been fixed for the next > build, so you can withFixedThreadPool(nThreads, threadFactory). But wouldn't it just be better to pass the ExecutorService then we can do it all simply? E.g. Executors.newFixedThreadPool(int nThreads, ThreadFactory threadFactory) Executors.newXXX > > The other thing is the default is awfull in our tests - as > it seems to > > create a new Thread in the pool for each AsyncDatagramChannel (for > > 100 multicasts received there were 130+ java threads). > Is this the default group or your own group? Also, is this > 100 outstanding receive operations on one/many channels or > 100 receives (one after the other)? One thing to say about > AsynchronousDatagramChannel is that it is currently just a > simple implementation and isn't plumbed into the event port > mechanism yet. Sorry I missused the word "default". I was using the following: pool = Executors.newCachedThreadPool(); asynchronousChannelGroup = AsynchronousChannelGroup.withCachedThreadPool(pool, 1); This is 100 unique sockets with one multicast group per socket. (we are currently stressing the test with 5Mbit/s per multicast group) However we just re ran with the actual default channel group by using AsynchronousDatagramChannel.open() And it creates even more threads!! > This hasn't been a problem so far because the > number of AsynchronousDatagramChannel is expected to be > small. I assume you don't observed this with the other > network channels as they work very differently. Can you define small? We will be targetting 100+ :-) > > Also isn't it the job of the Executor to choose chached > pooled or non > > pooled not the AsyncChannelGroup? > The default group uses a cached thread because it may be > shared by several applications. For other groups then you > supply the thread pool. But that is my point - you have to supply the pool (via the executor) *and also* to the AsynchronousChannelGroup. E.g. for a fixed pool you need the following: ExecutorService executor = Executors.newFixedThreadPool(8); AsynchronousChannelGroup group = AsynchronousChannelGroup.withFixedThreadPool(executor, 8); And for a cached pool you need: ExecutorService executor = Executors.newCachedThreadPool(); AsynchronousChannelGroup group = AsynchronousChannelGroup.withCachedThreadPool(executor, 4); Why don't we just have ExecutorService executor = // whatever executor type you want / fixed cached etc... AsynchronousChannelGroup group = AsynchronousChannelGroup.withThreadPool(executor); Does the group really care if its fixed or cached - doesn't it just want something to do some work? /James ********************************************************************************************************* This e-mail is confidential, the property of NDS Ltd and intended for the addressee only. Any dissemination, copying or distribution of this message or any attachments by anyone other than the intended recipient is strictly prohibited. If you have received this message in error, please immediately notify the postmaster at nds.com and destroy the original message. Messages sent to and from NDS may be monitored. NDS cannot guarantee any message delivery method is secure or error-free. Information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. We do not accept responsibility for any errors or omissions in this message and/or attachment that arise as a result of transmission. You should carry out your own virus checks before opening any attachment. Any views or opinions presented are solely those of the author and do not necessarily represent those of NDS. To protect the environment please do not print this e-mail unless necessary. NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 ********************************************************************************************************** From JNord at nds.com Thu Dec 18 02:35:40 2008 From: JNord at nds.com (Nord, James) Date: Thu, 18 Dec 2008 10:35:40 -0000 Subject: Thread pooling in AIO? In-Reply-To: <4947F0D9.1010309@redhat.com> References: <4947F0D9.1010309@redhat.com> Message-ID: > On 12/16/2008 11:48 AM, Nord, James wrote: > > The other thing is the default is awfull in our tests - as > it seems to > > create a new Thread in the pool for each AsyncDatagramChannel (for > > 100 multicasts received there were 130+ java threads). > > That's the way that fixed thread pools work. The core size > is equal to the pool size. Do you not mean a cached pool? >If ThreadPoolExecutor.execute() > is called and the number of threads is below the core size, > then a new thread is created rather than reusing an old one. > I think a fixed pool is probably the wrong choice for async > I/O - it would make more sense to use a cached pool which can > scale up on demand, with a sensible timeout for non-core threads. Whilst hundreds of threads may work well on NagraII boxes most intels have 8/16 or so cores you just end up in context swapping and contentending for resources which hurts. > One must also consider the task rejection policy - I don't > think it would be a good thing for async I/O tasks to be > rejected since the caller would have no idea what happened to > the operation. It would just "disappear". Is it rejected or does it block? I would say it should block. If this ends up down the line of having packet loss then that's in my opinion fair game. But what is the relationship between the native AIO layer and the threads we have here? At the native layer any thread can call GetQueuedCompletionStatus on the same IOCompletionPort... Are the threads we supply going to be in GetQueuedCompletionStatus until we have some data at which point they pop back into java land and invoke the CompletionHandler? In which case tasks should not be rejected - and the only possibility is we can not handle the data fast enough. But with one thread per cpu core we could not really run any faster if we tried. /James ********************************************************************************************************* This e-mail is confidential, the property of NDS Ltd and intended for the addressee only. Any dissemination, copying or distribution of this message or any attachments by anyone other than the intended recipient is strictly prohibited. If you have received this message in error, please immediately notify the postmaster at nds.com and destroy the original message. Messages sent to and from NDS may be monitored. NDS cannot guarantee any message delivery method is secure or error-free. Information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. We do not accept responsibility for any errors or omissions in this message and/or attachment that arise as a result of transmission. You should carry out your own virus checks before opening any attachment. Any views or opinions presented are solely those of the author and do not necessarily represent those of NDS. To protect the environment please do not print this e-mail unless necessary. NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 ********************************************************************************************************** From JNord at nds.com Thu Dec 18 03:53:29 2008 From: JNord at nds.com (Nord, James) Date: Thu, 18 Dec 2008 11:53:29 -0000 Subject: Thread pooling in AIO? In-Reply-To: <4947F667.8050901@sun.com> References: <4947F0D9.1010309@redhat.com> <4947F667.8050901@sun.com> Message-ID: > > I think a fixed pool is probably the wrong choice for async > I/O - it > > would make more sense to use a cached pool which can scale up on > > demand, with a sensible timeout for non-core threads. > If the server is completely event driven then a fixed thread > pool can work very well. > > One word of warning is one needs to be careful to avoid idle > timeout/keep alive on Windows due to the way that I/O works > on that platform. I/O operations are tied in the kernel to > the initiating thread; if a thread terminates (because it is > idle) then outstanding I/O operations that it has initiated > may be aborted. So your using AlertableIO[1] and not IOCompletion[2] ports then? Doesn't this have inherant scalability issues as you can only proccess on the same thread? It also means you can have some intersting thread management issues... [1]http://msdn.microsoft.com/en-us/library/aa363772(VS.85).aspx [2]http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx Regards, /James ********************************************************************************************************* This e-mail is confidential, the property of NDS Ltd and intended for the addressee only. Any dissemination, copying or distribution of this message or any attachments by anyone other than the intended recipient is strictly prohibited. If you have received this message in error, please immediately notify the postmaster at nds.com and destroy the original message. Messages sent to and from NDS may be monitored. NDS cannot guarantee any message delivery method is secure or error-free. Information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. We do not accept responsibility for any errors or omissions in this message and/or attachment that arise as a result of transmission. You should carry out your own virus checks before opening any attachment. Any views or opinions presented are solely those of the author and do not necessarily represent those of NDS. To protect the environment please do not print this e-mail unless necessary. NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 ********************************************************************************************************** From Alan.Bateman at Sun.COM Thu Dec 18 03:58:26 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 18 Dec 2008 11:58:26 +0000 Subject: Thread pooling in AIO/ In-Reply-To: References: <4947F3C8.3060101@sun.com> Message-ID: <494A3AE2.60300@sun.com> Nord, James wrote: > : > Sorry I missused the word "default". I was using the following: > pool = Executors.newCachedThreadPool(); > asynchronousChannelGroup = > AsynchronousChannelGroup.withCachedThreadPool(pool, 1); > > This is 100 unique sockets with one multicast group per socket. > > (we are currently stressing the test with 5Mbit/s per multicast group) > > However we just re ran with the actual default channel group by using > AsynchronousDatagramChannel.open() > And it creates even more threads!! > I haven't had cycles yet to swap in a replacement for the AsynchronousDatagramChannel implementation yet. The implementation that is there now is using a simple task per I/O operation. It needs to be replaced so that it uses overlapped I/O and the completion port mechanism like has already been done for the other network channels. The reason for the current simple implementation is to allow for feedback and test development. Although simple it isn't too bad as the number of UDP sockets is typically very small (using 100+ UDP ports is probably a lot more than average but still not a problem). The reason you are seeing slightly more threads in the default group is that it tries to deliver good performance and at the same time potentially serve many masters. It does this by dedicating a few threads to the I/O port and then creating threads on demand as required. As AsynchronousDatagramChannel is simply submitting tasks to the thread pool it means the I/O threads are never used (but you see them in the thread dump). Although the AsynchronousDatagramChannel is simple I wouldn't expect the performance to be too bad. Is it keeping up with the multicast load or are you seeing packet loss? I recall you were trying out the multicast support in DatagramChannel so I'm curious if this was the same load. > : > But that is my point - you have to supply the pool (via the executor) > *and also* to the AsynchronousChannelGroup. > > E.g. for a fixed pool you need the following: > ExecutorService executor = Executors.newFixedThreadPool(8); > AsynchronousChannelGroup group = > AsynchronousChannelGroup.withFixedThreadPool(executor, 8); > > And for a cached pool you need: > ExecutorService executor = Executors.newCachedThreadPool(); > AsynchronousChannelGroup group = > AsynchronousChannelGroup.withCachedThreadPool(executor, 4); > > > Why don't we just have > ExecutorService executor = // whatever executor type you want / > fixed cached etc... > AsynchronousChannelGroup group = > AsynchronousChannelGroup.withThreadPool(executor); > Others have also commented on this so there is a withThreadPool(executor) in for the next build. It's identical to withCachedThreadPool(executor,0) as this method can be invoked with any executor service. > Does the group really care if its fixed or cached - doesn't it just want > something to do some work? > This page attempts to explain it: http://openjdk.java.net/projects/nio/resources/AsynchronousIo.html -Alan. From Alan.Bateman at Sun.COM Thu Dec 18 04:07:38 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 18 Dec 2008 12:07:38 +0000 Subject: Thread pooling in AIO? In-Reply-To: References: <4947F0D9.1010309@redhat.com> <4947F667.8050901@sun.com> Message-ID: <494A3D0A.7060900@sun.com> Nord, James wrote: > : > > So your using AlertableIO[1] and not IOCompletion[2] ports then? > Doesn't this have inherant scalability issues as you can only proccess > on the same thread? > It also means you can have some intersting thread management issues... > > [1]http://msdn.microsoft.com/en-us/library/aa363772(VS.85).aspx > [2]http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx > On Windows the implementation uses overlapped I/O and completion ports for both file and socket I/O (except for AsynchronousDatagramChannel). On Windows Server 2003 and 2008 we regularly test with 20k+ connections. -Alan. From JNord at nds.com Thu Dec 18 04:15:23 2008 From: JNord at nds.com (Nord, James) Date: Thu, 18 Dec 2008 12:15:23 -0000 Subject: Thread pooling in AIO? In-Reply-To: <494A3D0A.7060900@sun.com> References: <4947F0D9.1010309@redhat.com> <4947F667.8050901@sun.com> <494A3D0A.7060900@sun.com> Message-ID: > > > > So your using AlertableIO[1] and not IOCompletion[2] ports then? > > Doesn't this have inherant scalability issues as you can > only proccess > > on the same thread? > > It also means you can have some intersting thread > management issues... > > > > [1]http://msdn.microsoft.com/en-us/library/aa363772(VS.85).aspx > > [2]http://msdn.microsoft.com/en-us/library/aa365198(VS.85).aspx > > > On Windows the implementation uses overlapped I/O and > completion ports for both file and socket I/O (except for > AsynchronousDatagramChannel). > On Windows Server 2003 and 2008 we regularly test with 20k+ > connections. Then you don't have to worry about which thread is handling the completion routine. It is only AlertableIO that has this limitation. from [1] "However, alertable I/O returns the result of the I/O request only to the thread that initiated it. I/O completion ports do not have this limitation." ********************************************************************************************************* This e-mail is confidential, the property of NDS Ltd and intended for the addressee only. Any dissemination, copying or distribution of this message or any attachments by anyone other than the intended recipient is strictly prohibited. If you have received this message in error, please immediately notify the postmaster at nds.com and destroy the original message. Messages sent to and from NDS may be monitored. NDS cannot guarantee any message delivery method is secure or error-free. Information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. We do not accept responsibility for any errors or omissions in this message and/or attachment that arise as a result of transmission. You should carry out your own virus checks before opening any attachment. Any views or opinions presented are solely those of the author and do not necessarily represent those of NDS. To protect the environment please do not print this e-mail unless necessary. NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 ********************************************************************************************************** From JNord at nds.com Thu Dec 18 04:34:33 2008 From: JNord at nds.com (Nord, James) Date: Thu, 18 Dec 2008 12:34:33 -0000 Subject: Thread pooling in AIO/ In-Reply-To: <494A3AE2.60300@sun.com> References: <4947F3C8.3060101@sun.com> <494A3AE2.60300@sun.com> Message-ID: > Although the AsynchronousDatagramChannel is simple I wouldn't > expect the performance to be too bad. Is it keeping up with > the multicast load or are you seeing packet loss? I recall > you were trying out the multicast support in DatagramChannel > so I'm curious if this was the same load. It is keeping up - but with CachedPool the load is much higher than with plain old java.net and much higher than the synchronous nio2. With a fixedPool (size of 8 on dual quad core box) the load is slightly higher than java.net and synchronous nio2. We are seeing the occaisonal lost packets with Async IO - but we're experimenting to find the best tradeoff of load and packet loss. > > But that is my point - you have to supply the pool (via the > executor) > > *and also* to the AsynchronousChannelGroup. > > > > E.g. for a fixed pool you need the following: > > ExecutorService executor = Executors.newFixedThreadPool(8); > > AsynchronousChannelGroup group = > > AsynchronousChannelGroup.withFixedThreadPool(executor, 8); > > > > And for a cached pool you need: > > ExecutorService executor = Executors.newCachedThreadPool(); > > AsynchronousChannelGroup group = > > AsynchronousChannelGroup.withCachedThreadPool(executor, 4); > > > > > > Why don't we just have > > ExecutorService executor = // whatever executor type > you want / fixed > > cached etc... > > AsynchronousChannelGroup group = > > AsynchronousChannelGroup.withThreadPool(executor); > > > Others have also commented on this so there is a > withThreadPool(executor) in for the next build. It's identical to > withCachedThreadPool(executor,0) as this method can be > invoked with any executor service. > > Does the group really care if its fixed or cached - doesn't it just > > want something to do some work? > > > This page attempts to explain it: > http://openjdk.java.net/projects/nio/resources/AsynchronousIo.html OK. ********************************************************************************************************* This e-mail is confidential, the property of NDS Ltd and intended for the addressee only. Any dissemination, copying or distribution of this message or any attachments by anyone other than the intended recipient is strictly prohibited. If you have received this message in error, please immediately notify the postmaster at nds.com and destroy the original message. Messages sent to and from NDS may be monitored. NDS cannot guarantee any message delivery method is secure or error-free. Information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. We do not accept responsibility for any errors or omissions in this message and/or attachment that arise as a result of transmission. You should carry out your own virus checks before opening any attachment. Any views or opinions presented are solely those of the author and do not necessarily represent those of NDS. To protect the environment please do not print this e-mail unless necessary. NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 ********************************************************************************************************** From Alan.Bateman at Sun.COM Thu Dec 18 05:11:18 2008 From: Alan.Bateman at Sun.COM (Alan Bateman) Date: Thu, 18 Dec 2008 13:11:18 +0000 Subject: Thread pooling in AIO/ In-Reply-To: References: <4947F3C8.3060101@sun.com> <494A3AE2.60300@sun.com> Message-ID: <494A4BF6.7040401@sun.com> Nord, James wrote: > : > It is keeping up - > but with CachedPool the load is much higher than with plain old java.net > and much higher than the synchronous nio2. > With a fixedPool (size of 8 on dual quad core box) the load is slightly > higher than java.net and synchronous nio2. > > We are seeing the occaisonal lost packets with Async IO - but we're > experimenting to find the best tradeoff of load and packet loss. > Thanks - the additional load is not a surprise (because the "simple" AsynchronousDatagramSocket is using select which performs very poorly on Windows). It is a surprise that you see any difference between the two thread pools. I assume it is because your cached thread pool is supporting 100 concurrent reads whereas your fixed thread pool is doing only 8 concurrent reads. -Alan. From JNord at nds.com Thu Dec 18 05:50:26 2008 From: JNord at nds.com (Nord, James) Date: Thu, 18 Dec 2008 13:50:26 -0000 Subject: Browsable source repo? Message-ID: Hi all, Is there a browsable source repo that give me a view of the "trunk". Similar to viewVC or ViewSVN etc... Or is there something I can do on http://hg.openjdk.java.net/nio/nio to give me a browsable directory structure and downloadable source files? /James ********************************************************************************************************* This e-mail is confidential, the property of NDS Ltd and intended for the addressee only. Any dissemination, copying or distribution of this message or any attachments by anyone other than the intended recipient is strictly prohibited. If you have received this message in error, please immediately notify the postmaster at nds.com and destroy the original message. Messages sent to and from NDS may be monitored. NDS cannot guarantee any message delivery method is secure or error-free. Information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. We do not accept responsibility for any errors or omissions in this message and/or attachment that arise as a result of transmission. You should carry out your own virus checks before opening any attachment. Any views or opinions presented are solely those of the author and do not necessarily represent those of NDS. To protect the environment please do not print this e-mail unless necessary. NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 ********************************************************************************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20081218/d9c022c8/attachment.html From hjort.dennis at gmail.com Thu Dec 18 13:02:03 2008 From: hjort.dennis at gmail.com (Dennis Hjort) Date: Thu, 18 Dec 2008 22:02:03 +0100 Subject: Browsable source repo? In-Reply-To: References: Message-ID: <514a207b0812181302q7b923811maf2e7cdd70d9454f@mail.gmail.com> Hi James, The answer you are looking for is in the Developers Guide [1]. There you will find instructions on how to install mercurial (similar to SVN), and how to check-out a clone of the repository. With kind regards // D [1] http://openjdk.java.net/guide/ 2008/12/18 Nord, James > Hi all, > > Is there a browsable source repo that give me a view of the "trunk". > Similar to viewVC or ViewSVN etc... > > Or is there something I can do on http://hg.openjdk.java.net/nio/nio to > give me a browsable directory structure and downloadable source files? > > /James > > > ********************************************************************************************************* > This e-mail is confidential, the property of NDS Ltd and intended for the > addressee only. Any dissemination, copying or distribution of this message > or any attachments by anyone other than the intended recipient is strictly > prohibited. If you have received this message in error, please immediately > notify the postmaster at nds.com and destroy the original message. Messages > sent to and from NDS may be monitored. NDS cannot guarantee any message > delivery method is secure or error-free. Information could be intercepted, > corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. > We do not accept responsibility for any errors or omissions in this message > and/or attachment that arise as a result of transmission. You should carry > out your own virus checks before opening any attachment. Any views or > opinions presented are solely those of the author and do not necessarily > represent those of NDS. > > To protect the environment please do not print this e-mail unless > necessary. > > NDS Limited Registered office: One Heathrow Boulevard, 286 Bath Road, West > Drayton, Middlesex, UB7 0DQ, United Kingdom. A company registered in England > and Wales Registered no. 3080780 VAT no. GB 603 8808 40-00 > > ********************************************************************************************************** > -- Dennis Hjort http://hack.org/dennis/ Home machine (IPv6 only): http://fc.hack.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20081218/6a045ac9/attachment.html