RFR (L): 8046148: JEP 158 Unified JVM Logging
vitalyd at gmail.com
Thu Sep 10 00:47:17 UTC 2015
It sounds like you're proposing a hierarchical naming scheme for log
statements; if anyone is familiar with the Graphite tool, it does something
similar for metric names. You can then wildcard certain parts of the path
when using its query tools. Is that what you have in mind?
That's flexible but can lead to other issues. Every log line will need to
know which path it's using, which can lead to an explosion of these paths
as JVM devs will need to know what tags already exist for the
(sub)component they're logging from. It's much easier to quickly decide
what level is appropriate. Consumers will still need to know what tags
exist to get the right level of logging; wildcarding parts may be too much,
fully specifying all desired tags can become unwieldy.
Before going further, I'll stop to confirm that this is your gist.
As for Chronicle, that seems like a separate issue altogether which is the
mechanics of the logging. I do agree that logging needs to be as efficient
as possible as it's done synchronously. This also implies the filtering
scheme has to be efficient when the JVM decides whether the log should be
emitted or not.
sent from my phone
On Sep 9, 2015 7:31 PM, "Kirk Pepperdine" <kirk at jclarity.com> wrote:
> > On Sep 9, 2015, at 10:27 AM, Marcus Larsson <marcus.larsson at oracle.com>
> > Hi,
> > On 2015-09-08 14:47, Kirk Pepperdine wrote:
> >>> On Sep 8, 2015, at 1:42 PM, Claes Redestad <claes.redestad at oracle.com>
> >>> On 2015-09-08 12:45, Kirk Pepperdine wrote:
> >>>> Hi,
> >>>> Indeed, not all messages are hierarchical. This is one of the reasons
> I originally questioned the use of log levels. For example, debugging and
> informational logs may come from the same source and I certainly wouldn’t
> want to be forced mix them to satisfy a forced structure. Is the intention
> you can use either or? I ask because I’m not sure that mixing them with
> tags will alleviate the problems I commonly see when tuning applications in
> the field.
> >>>>> Log levels have always been part of the plan as far as I can tell.
> Both the first and second proposal include the levels in their
> descriptions. In any case, the JEP is due for integration and it is too
> late for such a fundamental change to the feature now.
> >>>> I would like to believe that this work is important enough to get
> right and that it’s never too late for fundamental changes up until it is
> integrated. I would like to believe that there are back out strategies in
> the cases where implementations show unintended consequences.
> >>> Can you give an example of what you believe will go wrong with
> allowing a mix of levels and tags and how removing levels from the JEP
> would improve things?
> >> I’m slowly working through the patch but I’ve not had time to get
> through all of it yet so some things mentioned here might be in the source
> already… but here are some comments from experiences.
> >> As you have discovered, you can’t define a tag system using levels.
> That said, if you have tags you can define levels but you don’t need to
> explicitly support them, it just come for free out of the box. So I would
> argue both the spec and implementation would be simplified by moving to a
> tag system. More over, it doesn’t preclude someone defining a hierarchical
> system if them so need to have one.
> > While implementing levels using tags might simplify the implementation,
> I think it would introduce more complexity for the log producers, since
> there would be a much wider choice between "levels" in contrast to the
> current six levels.
> This actually doesn’t introduce a complexity, it simplifies things. Again,
> I think it would be advantageous to look at messaging implementations such
> as Chronicle.
> > It should be easy to specify how much logging you want, so I don't think
> it would be a good idea to omit levels completely either.
> tags do not preclude levels. If people want levels they can easily be
> created with tags. Again Neal Ford has a number of great talks on taxonomy
> and in my opinion he gets it spot on when he talks about the problems with
> level based systems.
> > For example, some consumers of the logging might only be interested in a
> certain level of log messages, but from any component or from a broad range
> of components. Others might want every piece of logging belonging to a
> (very) specific component.
> Again, tags do a much better job at this.
> >> Next, I think there are two points of view that you have to consider.
> Usability for the producers and usability for the consumers. From a
> consumer POV what I would like to see is a logging system that promotes
> stability in the output. Using broad ill defined categories as they are
> currently defined in the JDK have historically lead to unstable output.
> This in turn breaks down stream tooling that maybe used to help triage
> system out in the wild. How does this happen? Well what gets put into a
> level is completely arbitrary. What goes into warning, fine, finest.. who
> knows? More to the point, the producers often have to make a quick decision
> and in their cubicles they often don’t see the entire downstream tooling
> chains that they are affecting when they do this even when the information
> being added it very valuable. I’m going to suggest a system, maybe not
> *the* system or even the *best* system. It’s just a simple system that
> people have been using in pub/sub messaging systems for years. I think the
> technique applies because logging is a form of journalling which is just
> another form of messaging (hence I suggested people looking at Peter
> Lawyer’s Chronicle which is designed to be used in HFT applications). We
> can also use ObjectName (jmx) to define tags.
> >> If you have a tag, let say, gc.agetable, it’s pretty clear what you
> should get. Also that output is very likely to remain stable over time. In
> fact, my experience with PrintTenuringDistrubtion is just that. The output
> has remained stable for years. I cannot say that about any overly broad
> category of GC logging such as PrintGCDetails (which I would equate to a
> level). If I have another tag say, gc.collection.scavenge I should get
> information about scavenges. Again purpose is very clear from both a
> producer/consumer POV. I can take age data or scavenge or both or
> gc.collection.* to get all sub categories for gc.collection. This looks
> like levels and indeed could be used to implement levels but it’s based on
> categories (tags). At any rate, the taxonomy is clearer and should offer
> guidelines that makes it easier for the developer to push the information
> out on the right channel while offering better stability needed for
> downstream tooling.
> > The problem of classifying log messages exists whether you use real
> levels or not; "Should this log message be tagged with verbose?" vs "Should
> this log message be on trace level?”.
> I think the problem can be diminished in complexity when you include the
> vocabulary of the domain in your tag systems. If you think of forcing
> predefined terms then yes, you are correct.
> >>> Having high-level tags which interact with the standard "levels" seem
> like a natural fit to me, while the framework proposed in the JEP make it
> possible for each VM team to support ever more detailed tags to each
> subsystem where that might be appropriate. For better or worse it adds a
> degree of freedom. Is your concern that we need more granular tags going
> forward to help zooming in on specific issues, and that levels for such
> niche tags might seem/feel superfluous?
> >> If the system has been built with tags on levels then I would suggest
> the system has been built upside down or inside out, however you want to
> describe it.. in the same way that JDK logging was build upside down (first
> problem, super class knows about subclasses and that is a primary source of
> classloader leaks and that is just the first issue).
> >> Kind regards,
> >> Kirk
> > I would say the system has been built with levels on tags. The tags
> define *what* you want to log, and the levels define *how much* information
> you want. The levels are sort of a filter on the tags you select.
> So what you are saying is that as a producer I can completely ignore
> levels if I so choose to do so.
> I do have to apologize as I’ve not been able to get through the entire
> implementation yet but I hope to shortly.
More information about the hotspot-dev