RFR: Add NMT support for Java heap
per.liden at oracle.com
Thu Dec 14 15:22:45 UTC 2017
On 2017-12-12 14:29, Erik Österlund wrote:
> Hi Per,
> I suppose this does indeed depend on how you view things. To me, when
> there are different interpretations what should be reported, I try to
> understand why we are reporting it. If we report a number, then there
> should be some kind of question to which the reported number provides an
> The number you propose now is accurate in terms of what is called by
> mmap, but I do not know what useful question it answers. It will
> seemingly report ~17 TB always regardless of potentially interesting
> things such as, e.g. how close we are to hitting the roof of max memory
> that may be committed. In other words, I am not sure what useful
> question there is to which the answer is how much virtual address space
> has been reserved by the Java heap.
Given how NMT records/displays data, especially in "detail" mode where
it's not just one number but a ranges of reserved and committed, I think
we have two options here:
1) Register all heap views as reserved.
2) Register only one heap view as reserved.
Only registering "max heap size" as reserved wouldn't really work,
again, since it's a range and we don't necessarily later commit memory
within that reserved range.
None of the alternatives above are optimal, but at this time I think #1
is slightly better, since at least it can answer two questions
truthfully (the usefulness of those questions is debatable):
1) What part of the address spaces does the GC actually reserve for the
Java heap. Could be useful to know if you want to e.g. set VA limits
(think ulimit -v).
2) And, as I mentioned before, how does this or that line in
/proc/.../maps correlate to the JVM memory usage?
So, I'll suggest we do #1 for now. We'll change it if we find a good
reason to do so.
> On 2017-12-12 12:20, Per Liden wrote:
>> On 2017-12-12 11:50, Aleksey Shipilev wrote:
>>> On 12/12/2017 11:42 AM, Per Liden wrote:
>>>> As Aleksey noticed, we don't register the Java heap with the native
>>>> memory tracker. Here's a patch
>>>> to do that.
>>> Patch looks good, but the NMT "reserved" data is off the charts:
>>> Total: reserved=17180280855KB, committed=17143487KB
>>> - Java Heap (reserved=17179869184KB,
>>> (mmap: reserved=17179869184KB,
>>> I guess this should not pass ZAddressSpaceSize, and instead tell the
>>> reserved space of the first
>>> + // Register address space with native memory tracker
>>> + nmt_reserve(ZAddressSpaceStart, ZAddressSpaceSize);
>> I think this is correct actually. But it depends on how one views
>> things I guess. As I see it, I want to be able to look in
>> /proc/../maps and with NMT see what the different mappings correlate
>> to. If we only registered the first heap view, then there would be a
>> big mysterious reservation that would go unaccounted. That doesn't
>> sound right to me, but I'm open for hearing other opinions on this.
>> The big number there covers all addresses for all heap views/mappings
>> (i.e. the actual address space that is reserved). It should be noted
>> that, in ZGC, the heap address space doesn't have a 1:1 relation with
>> max heap size.
More information about the zgc-dev