RFR (L): 8046148: JEP 158 Unified JVM Logging
marcus.larsson at oracle.com
Wed Sep 9 11:26:31 UTC 2015
On 2015-09-09 11:31, Kirk Pepperdine wrote:
>> On Sep 9, 2015, at 10:27 AM, Marcus Larsson <marcus.larsson at oracle.com> wrote:
>> On 2015-09-08 14:47, Kirk Pepperdine wrote:
>>>> On Sep 8, 2015, at 1:42 PM, Claes Redestad <claes.redestad at oracle.com> wrote:
>>>> On 2015-09-08 12:45, Kirk Pepperdine wrote:
>>>>> Indeed, not all messages are hierarchical. This is one of the reasons I originally questioned the use of log levels. For example, debugging and informational logs may come from the same source and I certainly wouldn’t want to be forced mix them to satisfy a forced structure. Is the intention you can use either or? I ask because I’m not sure that mixing them with tags will alleviate the problems I commonly see when tuning applications in the field.
>>>>>> Log levels have always been part of the plan as far as I can tell. Both the first and second proposal include the levels in their descriptions. In any case, the JEP is due for integration and it is too late for such a fundamental change to the feature now.
>>>>> I would like to believe that this work is important enough to get right and that it’s never too late for fundamental changes up until it is integrated. I would like to believe that there are back out strategies in the cases where implementations show unintended consequences.
>>>> Can you give an example of what you believe will go wrong with allowing a mix of levels and tags and how removing levels from the JEP would improve things?
>>> I’m slowly working through the patch but I’ve not had time to get through all of it yet so some things mentioned here might be in the source already… but here are some comments from experiences.
>>> As you have discovered, you can’t define a tag system using levels. That said, if you have tags you can define levels but you don’t need to explicitly support them, it just come for free out of the box. So I would argue both the spec and implementation would be simplified by moving to a tag system. More over, it doesn’t preclude someone defining a hierarchical system if them so need to have one.
>> While implementing levels using tags might simplify the implementation, I think it would introduce more complexity for the log producers, since there would be a much wider choice between "levels" in contrast to the current six levels.
> This actually doesn’t introduce a complexity, it simplifies things. Again, I think it would be advantageous to look at messaging implementations such as Chronicle.
At least to me, a choice between one of six possible alternatives is
easier than a choice of any number of alternatives from an unlimited set.
>> It should be easy to specify how much logging you want, so I don't think it would be a good idea to omit levels completely either.
> tags do not preclude levels. If people want levels they can easily be created with tags. Again Neal Ford has a number of great talks on taxonomy and in my opinion he gets it spot on when he talks about the problems with level based systems.
>> For example, some consumers of the logging might only be interested in a certain level of log messages, but from any component or from a broad range of components. Others might want every piece of logging belonging to a (very) specific component.
> Again, tags do a much better job at this.
>>> Next, I think there are two points of view that you have to consider. Usability for the producers and usability for the consumers. From a consumer POV what I would like to see is a logging system that promotes stability in the output. Using broad ill defined categories as they are currently defined in the JDK have historically lead to unstable output. This in turn breaks down stream tooling that maybe used to help triage system out in the wild. How does this happen? Well what gets put into a level is completely arbitrary. What goes into warning, fine, finest.. who knows? More to the point, the producers often have to make a quick decision and in their cubicles they often don’t see the entire downstream tooling chains that they are affecting when they do this even when the information being added it very valuable. I’m going to suggest a system, maybe not *the* system or even the *best* system. It’s just a simple system that people have been using in pub/sub messaging systems for years. I think the technique applies because logging is a form of journalling which is just another form of messaging (hence I suggested people looking at Peter Lawyer’s Chronicle which is designed to be used in HFT applications). We can also use ObjectName (jmx) to define tags.
>>> If you have a tag, let say, gc.agetable, it’s pretty clear what you should get. Also that output is very likely to remain stable over time. In fact, my experience with PrintTenuringDistrubtion is just that. The output has remained stable for years. I cannot say that about any overly broad category of GC logging such as PrintGCDetails (which I would equate to a level). If I have another tag say, gc.collection.scavenge I should get information about scavenges. Again purpose is very clear from both a producer/consumer POV. I can take age data or scavenge or both or gc.collection.* to get all sub categories for gc.collection. This looks like levels and indeed could be used to implement levels but it’s based on categories (tags). At any rate, the taxonomy is clearer and should offer guidelines that makes it easier for the developer to push the information out on the right channel while offering better stability needed for downstream tooling.
>> The problem of classifying log messages exists whether you use real levels or not; "Should this log message be tagged with verbose?" vs "Should this log message be on trace level?”.
> I think the problem can be diminished in complexity when you include the vocabulary of the domain in your tag systems. If you think of forcing predefined terms then yes, you are correct.
With the current proposal one can easily direct VM wide warnings and
errors to a log file, the necessary configuration would be
-Xlog:all=warning:out.log. Removing the concept of levels and only using
domain specific tags to classify log messages would make it pretty
complicated to achieve the same effect. The consumer would have to know
about all the different components and their domain specific
warning/error-equivalent tags in order to get the same output, or am I
>>>> Having high-level tags which interact with the standard "levels" seem like a natural fit to me, while the framework proposed in the JEP make it possible for each VM team to support ever more detailed tags to each subsystem where that might be appropriate. For better or worse it adds a degree of freedom. Is your concern that we need more granular tags going forward to help zooming in on specific issues, and that levels for such niche tags might seem/feel superfluous?
>>> If the system has been built with tags on levels then I would suggest the system has been built upside down or inside out, however you want to describe it.. in the same way that JDK logging was build upside down (first problem, super class knows about subclasses and that is a primary source of classloader leaks and that is just the first issue).
>>> Kind regards,
>> I would say the system has been built with levels on tags. The tags define *what* you want to log, and the levels define *how much* information you want. The levels are sort of a filter on the tags you select.
> So what you are saying is that as a producer I can completely ignore levels if I so choose to do so.
Each message must be logged to a level, so it can't be ignored really. I
just don't see what you mean with the inside-out argument. Whether you
view it as levels on tags or tags on levels is up to you.
> I do have to apologize as I’ve not been able to get through the entire implementation yet but I hope to shortly.
More information about the hotspot-dev