[RFR]: Per thread IO statistics in JFR
gunter.haug at sap.com
Thu Jan 17 15:15:40 UTC 2019
> Today we have File Read, File Write, Socket Read, and Socket Write
> events. The hook points are added to the JDK using bytecode
> instrumentation. This happens when you start a JFR recording, so there
> is no overhead unless you use it.
However, in this context it's worth to note that the information provided by the events Erik describes is quite different from what we have proposed. The existing events are triggered only if the time the IO operation takes, exceeds a threshold (10ms/20ms for profile/default settings respectively). They are aimed for providing information on the time IO operations take and do not provide per thread IO statistics. What we are interested in is per thread information similar to the jdk.ThreadAllocationStatistics event.
With respect to performance impact, I was not able to observe a degradation above the measurement noise level in real-world benchmarks. I will try to do measurements with synthetic benchmarks and report back ASAP.
On 17.01.19, 13:07, "Erik Gahlin" <erik.gahlin at oracle.com> wrote:
On 2019-01-17 09:00, Alan Bateman wrote:
> On 17/01/2019 07:23, Thomas Stüfe wrote:
>> Do you object against keeping these counters (which basically boils
>> down to Thread::current->stat_structure->counter++)? Or do you even
>> object against making upcalls into the jvm? Note that, if deemed
>> necessary, we could omit updating the counters unless JFR or our
>> extended thread dumps are activated (which are the consumers of the
>> In any case, I would have assumed the costs for upcall + counter
>> update to be insignificant compared to the IO calls. We should of
>> course measure that.
>> If you generally object upcalls into the libjvm for
>> statistical/monitoring reasons, this would make matters on a number
>> of fronts more complicated. For instance, it was discussed extending
>> NMT coverage to the JDK - which is already in part reality at
>> Unsafe.AllocateMemory - and this would have to be done with upcalls too.
> There are many issues here that will need write-up and discussion,
> maybe a JEP if discussions converge on a proposal to bring into the
> main line as this is a significant change with implications for many
> areas of the platform. It also potentially conflicts in direction with
> some of the other projects in progress (particularly with Loom trying
> to re-imagine threads, do you really want to collect I/O stats on a
> per thread basis in the future???).
> As regards the points to instrument then I think we have to assume
> that much of the native code that is targeted by the current webrev
> will go away or change significantly in the future. We've been on that
> path for some time, e.g. the zip area or the prototype to replace the
> SocketImpl used for classic networking that eliminates a lot of the
> native code touched in that area by the webrev. Once Panama is further
> along then I assume we will want to make use of it in the core
> libraries and at least initially replace the JNI methods that just
> wrap syscalls today, and longer term more significant refactoring. My
> point is that instrumenting native methods may not be the right
> approach, instead maybe we should be look at instrumenting the I/O
> paths at the java level as that will likely play better with the VM.
> There is some support for collecting I/O stats in JFR today and maybe
> someone working in that area can explain that a bit more and what the
> issues are.
Today we have File Read, File Write, Socket Read, and Socket Write
events. The hook points are added to the JDK using bytecode
instrumentation. This happens when you start a JFR recording, so there
is no overhead unless you use it.
> It's impossible to tell from the mail with the webrev what has been
> explored and not explored. It feels like early stages in a much large
> project that will need a write up of prototypes before a direction can
> be proposed.
More information about the core-libs-dev