RFR(L) 8046598: Scalable Native memory tracking development
zhengyu.gu at oracle.com
Tue Jul 15 20:12:04 UTC 2014
This is an update to previous RFR 8028541: Native Memory Tracking
enhancement, the original one is closed as duplicate of current one.
The update is mainly based on feedback from Coleen and Christian:
- Refactored MemReporter to break up some large functions and
eliminate duplicated code.
- Minor change to MemBaseline for eliminating duplicated code.
- Changed MEMFLAGS type from unsigned short => MemoryType.
Also added unit tests for LinkedList.
The note from RFR 8028541:
This is significant rework of native memory tracking introduced in
The goal of this enhancement is to improve scalability, from both
tracking memory and CPU usage perspectives, so it can scale well with
increased memory allocation in large applications.
The enhancement is mainly focused on malloc memory tracking, whose
activities are several magnitude higher than virtual memory, and was the
main bottleneck in early implementation.
Instead of using book keeping records for tracking malloc activities,
new implementation co-locates tracking data along side with user data by
using a prefixed header. The header size is 8 bytes on 32-bit systems
and 16 bytes on 64-bit systems, which ensure that user data also align
Virtual memory tracking still uses book keeping records, and
ThreadCritical lock is always acquired to alter the records and related
Summary tracking data is maintained in static data structures, via
atomic operations. Malloc detail tracking call stacks are maintained in
a lock free hashtable.
The key improvements:
1. Up-to-date tracking report.
2. Detail tracking now shows multiple call frames. Number of frames
is compilation time decision, currently default to 4.
3. Malloc tracking is lock free.
4. Tracking summary is reported in hs_err file when native memory
tracking is enabled.
5. Query is faster, uses little memory and need a very little process.
The drawback is that, malloc tracking header is always needed if native
memory tracking has ever been enabled, even after tracking is shutdown.
The most noticeable impact for JVM developers, is that Arena now also
take memory type as constructor parameter, besides the new operators.
Arena* a = new (mtCode) Arena() => Arena* a = new (mtCode)
The webrev shows modification of about 60 files, but most of them are
due to tracking API changes, mainly due to tracking stack, now, is an
object, vs. a single pc.
The most important files for this implementations are:
mallocTracker.hpp/cpp and mallocTracker.inline.hpp
- NMT test suite
- Kitchensink stability test for 16+ days
More information about the hotspot-dev