[PATCH] Linux NUMA support for HotSpot

Andi Kleen andi at firstfloor.org
Mon Mar 3 05:32:05 PST 2008

On Mon, Mar 03, 2008 at 04:25:53PM +0300, Igor Veresov wrote:
> On Monday 03 March 2008 15:57:19 Andi Kleen wrote:
> > On Mon, Mar 03, 2008 at 03:19:13PM +0300, Igor Veresov wrote:
> > > > > all working quite well, actually.
> > > >
> > > > One obvious issue I found in the Solaris code was that it neither
> > > > binds threads to nodes (afaik), nor tries to keep up with a thread
> > > > migrating to another node. It just assumes always the same thread:node
> > > > mapping which surely cannot be correct?
> > >
> > > It is however correct. Solaris assigns a home locality group (a node) to
> > > each lwp. And while lwps can be temporary migrated to a remote group,
> > > page allocation still happens on a home node and the lwp is predominantly
> > > scheduled to run in its home lgroup. For more information you could refer
> > > to
> >
> > Interesting. I tried a similar scheduling algorithm on Linux a long time
> > ago (it was called the "homenode scheduler") and it was a general loss on
> > my testing on smaller AMD systems. But maybe Solaris does it all different.
> >
> > Anyways on Linux that won't work because it doesn't have the concept
> > of a homenode.
> Yes, but it has static memory binding instead, which alleviates this problem.

That would require statically binding the threads too which is by default not
a good idea without explicit user configuration

The reasoning is that not using a CPU is always worse than using 
remote memory at least on systems with reasonable small NUMA factor.

(that is what killed the homenode scheduler too) 

> >
> > The other problems is that it seemed to always assume all the threads
> > will consume the whole system and set up for all nodes, which seemed dodgy.
> You mean the allocator? Actually it is adaptive to the allocation rate on a 
> node, which in effect makes the whole eden space usable for applications with 
> asymmetric per-thread allocation rate. This of course also helps with the 
> case when the number of threads is less than the number of nodes.

It didn't seem to adapt though. Or maybe I'm misremembering the code,
it was some time ago.


More information about the hotspot-dev mailing list