From shade at redhat.com Fri Jul 2 07:56:36 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 2 Jul 2021 09:56:36 +0200 Subject: Bug metadata updates for 11u bugs Message-ID: Hi, I have a question about bug metadata handling. Consider this backport history: https://bugs.openjdk.java.net/browse/JDK-8268891 It was pushed to 11.0.13. Bots updated "Resolved In Build" to "team". Then yesterday Saravana updated "Resolved In Build" to "b01". It does seem the sibling backport to "11.0.13-oracle" got updated to "b01" at the same time. Does this reflect the tag in Oracle internal tree only? Or is it a bug metadata update hiccup, i.e. the scripts updated both 11.0.13-oracle (correctly) and 11.0.13 (unnecessarily)? -- Thanks, -Aleksey From sgehwolf at redhat.com Fri Jul 2 08:15:39 2021 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 02 Jul 2021 10:15:39 +0200 Subject: Bug metadata updates for 11u bugs In-Reply-To: References: Message-ID: On Fri, 2021-07-02 at 09:56 +0200, Aleksey Shipilev wrote: > Hi, > > I have a question about bug metadata handling. Consider this backport > history: > ?? https://bugs.openjdk.java.net/browse/JDK-8268891 > > It was pushed to 11.0.13. Bots updated "Resolved In Build" to "team". > Then yesterday Saravana > updated "Resolved In Build" to "b01". It does seem the sibling > backport to "11.0.13-oracle" got > updated to "b01" at the same time. > > Does this reflect the tag in Oracle internal tree only? Or is it a > bug metadata update hiccup, i.e. > the scripts updated both 11.0.13-oracle (correctly) and 11.0.13 > (unnecessarily)? Yes, I've noticed this too on a variety of bugs. Please don't change the bug metadata of OpenJDK 11 bugs (11.0.XX *not* 11.0.XX-oracle). There is no OpenJDK 11.0.13+1 tag yet nor a build. It's only resolved in the team repo, which corresponds to jdk11u-dev git tree. For the ones I've noticed I've changed it back. Thanks, Severin From Alan.Bateman at oracle.com Fri Jul 2 08:25:29 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 2 Jul 2021 09:25:29 +0100 Subject: Bug metadata updates for 11u bugs In-Reply-To: References: Message-ID: <239121ef-d532-e70e-558f-2d8895fff79e@oracle.com> On 02/07/2021 08:56, Aleksey Shipilev wrote: > Hi, > > I have a question about bug metadata handling. Consider this backport > history: > ? https://bugs.openjdk.java.net/browse/JDK-8268891 > > It was pushed to 11.0.13. Bots updated "Resolved In Build" to "team". > Then yesterday Saravana updated "Resolved In Build" to "b01". It does > seem the sibling backport to "11.0.13-oracle" got updated to "b01" at > the same time. > > Does this reflect the tag in Oracle internal tree only? Or is it a bug > metadata update hiccup, i.e. the scripts updated both 11.0.13-oracle > (correctly) and 11.0.13 (unnecessarily)? > This looks like a mistake rather than intentional, maybe a script doing bulk updates when an Oracle JDK build was promoted. Hopefully Saravana can fix it and the backport issues can be restored. -Alan From christoph.langer at sap.com Fri Jul 2 09:02:42 2021 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 2 Jul 2021 09:02:42 +0000 Subject: Bug metadata updates for 11u bugs In-Reply-To: <239121ef-d532-e70e-558f-2d8895fff79e@oracle.com> References: <239121ef-d532-e70e-558f-2d8895fff79e@oracle.com> Message-ID: Hi, I've reported this to ops yesterday already and Kevin Rushforth responded that they'll have a look. After all, I think no corrective action is necessary because all those bugs will be promoted to b01 as soon as we tag OpenJDK 11.0.13+1 anyway. It's just invalid for the moment... Best regards Christoph > -----Original Message----- > From: jdk-dev On Behalf Of Alan Bateman > Sent: Freitag, 2. Juli 2021 10:25 > To: Aleksey Shipilev ; jdk-dev at openjdk.java.net > Subject: Re: Bug metadata updates for 11u bugs > > On 02/07/2021 08:56, Aleksey Shipilev wrote: > > Hi, > > > > I have a question about bug metadata handling. Consider this backport > > history: > > ? https://bugs.openjdk.java.net/browse/JDK-8268891 > > > > It was pushed to 11.0.13. Bots updated "Resolved In Build" to "team". > > Then yesterday Saravana updated "Resolved In Build" to "b01". It does > > seem the sibling backport to "11.0.13-oracle" got updated to "b01" at > > the same time. > > > > Does this reflect the tag in Oracle internal tree only? Or is it a bug > > metadata update hiccup, i.e. the scripts updated both 11.0.13-oracle > > (correctly) and 11.0.13 (unnecessarily)? > > > This looks like a mistake rather than intentional, maybe a script doing > bulk updates when an Oracle JDK build was promoted. Hopefully Saravana > can fix it and the backport issues can be restored. > > -Alan From praveen.mohan at oracle.com Fri Jul 2 11:05:11 2021 From: praveen.mohan at oracle.com (Praveen Mohan) Date: Fri, 2 Jul 2021 11:05:11 +0000 Subject: Bug metadata updates for 11u bugs Message-ID: Hi All, We have inadvertently updated all the 11.0.13 records while updating 11.0.13-oracle backports for 'Resolved in Build'. We have reverted the change on all such records to what was there prior to our update. Thanks to Severin for reverting some of them already. We will fix our scripts appropriately. Sorry for the inconvenience. Thanks, Praveen > -----Original Message----- > From: jdk-dev > On Behalf Of Alan Bateman > Sent: Freitag, 2. Juli 2021 10:25 > To: Aleksey Shipilev >; jdk-dev at openjdk.java.net > Subject: Re: Bug metadata updates for 11u bugs > > On 02/07/2021 08:56, Aleksey Shipilev wrote: > > Hi, > > > > I have a question about bug metadata handling. Consider this backport > > history: > > https://bugs.openjdk.java.net/browse/JDK-8268891 > > > > It was pushed to 11.0.13. Bots updated "Resolved In Build" to "team". > > Then yesterday Saravana updated "Resolved In Build" to "b01". It does > > seem the sibling backport to "11.0.13-oracle" got updated to "b01" at > > the same time. > > > > Does this reflect the tag in Oracle internal tree only? Or is it a bug > > metadata update hiccup, i.e. the scripts updated both 11.0.13-oracle > > (correctly) and 11.0.13 (unnecessarily)? > > > This looks like a mistake rather than intentional, maybe a script doing > bulk updates when an Oracle JDK build was promoted. Hopefully Saravana > can fix it and the backport issues can be restored. > > -Alan From thomas.schatzl at oracle.com Thu Jul 8 16:00:39 2021 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 8 Jul 2021 18:00:39 +0200 Subject: New JDK Reviewer: Leo Korinth Message-ID: <74a3aaf9-9c5a-22a7-fb11-d45b6c9dc433@oracle.com> Voting for Leo Korinth [1] is now closed. Yes: 28 Veto: 0 Abstain: 0 According to the Bylaws definition of Three-Vote Consensus, this is sufficient to approve the nomination. Thomas Schatzl [1] https://mail.openjdk.java.net/pipermail/jdk-dev/2021-June/005710.html From matthias.baesken at sap.com Fri Jul 9 12:55:30 2021 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 9 Jul 2021 12:55:30 +0000 Subject: github presubmit tests linux x64 : 'g++-10' was not found Message-ID: Hi, I get this strange error in the github presubmit tests on linux x64 : https://github.com/MBaesken/jdk/runs/3028479786 E: Version '10.2.0-5ubuntu1~20.04' for 'gcc-10' was not found 46E: Version '10.2.0-5ubuntu1~20.04' for 'g++-10' was not found 47Error: Process completed with exit code 100. Looks like an infra issue, could someone have a look please ? Thanks, Matthias From shade at redhat.com Fri Jul 9 13:04:42 2021 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 9 Jul 2021 15:04:42 +0200 Subject: github presubmit tests linux x64 : 'g++-10' was not found In-Reply-To: References: Message-ID: <63a00a99-0872-ace7-d1e3-9b714643fce9@redhat.com> On 7/9/21 2:55 PM, Baesken, Matthias wrote: > Looks like an infra issue, could someone have a look please ? It was fixed a few weeks ago, make sure you have updated master: https://bugs.openjdk.java.net/browse/JDK-8269148 -- Thanks, -Aleksey From mark.reinhold at oracle.com Mon Jul 12 15:08:47 2021 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 12 Jul 2021 08:08:47 -0700 (PDT) Subject: JDK 17 enters Rampdown Phase Two later this week Message-ID: <20210712150847.E9E553EB431@eggemoggin.niobe.net> Reminder: JDK 17 will enter Rampdown Phase Two this Thursday, 15 July, at 15:00 UTC [1]. In the meantime, if you?re responsible for any of the bugs on the RDP 1 candidate-bug list [2] then please see JEP 3 for guidance on how to handle them. Once we?re in RDP 2 we will turn our focus to P1 and P2 bugs only. - Mark [1] https://time.is/1500_15_July_2021_in_UTC/Stockholm/London/Boston/San_Francisco?JDK_17_Rampdown_Phase_Two [2] https://j.mp/jdk-rdp-2 From adinn at redhat.com Tue Jul 13 12:21:46 2021 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 13 Jul 2021 13:21:46 +0100 Subject: CFV: New JDK Reviewer: Nick Gasson Message-ID: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> I hereby nominate Nick Gasson to JDK Reviewer. Nick has been contributing to the JDK project since 2018 and has a very solid understanding of the AArch64 port. He has contributed 59 commits to the main JDK repository fixing significant Arch64-related problems across the JDK code base including the compilers, GCs, JVM core/runtime and build system [1]. Nick has also recently taken it upon himself to start contributing AArch64 fixes to the Valhalla project adding a further 7 commits [2]. His wide ranging interest and understanding will make him a very valuable JDK reviewer. Votes are due by 23:00 GMT 27 July 2021. Only current JDK Reviewers [3] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Three-Vote Consensus voting instructions, see [4]. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill [1] https://github.com/openjdk/jdk/commits?author=nick-arm [2] https://github.com/openjdk/valhalla/commits?author=nick-arm [3] https://openjdk.java.net/census [4] https://openjdk.java.net/projects/#reviewer-vote From martin.doerr at sap.com Tue Jul 13 12:52:18 2021 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 13 Jul 2021 12:52:18 +0000 Subject: AW: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes Best regards, Martin Von: jdk-dev im Auftrag von Andrew Dinn Datum: Dienstag, 13. Juli 2021 um 14:23 An: jdk-dev Betreff: CFV: New JDK Reviewer: Nick Gasson I hereby nominate Nick Gasson to JDK Reviewer. Nick has been contributing to the JDK project since 2018 and has a very solid understanding of the AArch64 port. He has contributed 59 commits to the main JDK repository fixing significant Arch64-related problems across the JDK code base including the compilers, GCs, JVM core/runtime and build system [1]. Nick has also recently taken it upon himself to start contributing AArch64 fixes to the Valhalla project adding a further 7 commits [2]. His wide ranging interest and understanding will make him a very valuable JDK reviewer. Votes are due by 23:00 GMT 27 July 2021. Only current JDK Reviewers [3] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Three-Vote Consensus voting instructions, see [4]. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill [1] https://github.com/openjdk/jdk/commits?author=nick-arm [2] https://github.com/openjdk/valhalla/commits?author=nick-arm [3] https://openjdk.java.net/census [4] https://openjdk.java.net/projects/#reviewer-vote From zgu at redhat.com Tue Jul 13 13:21:47 2021 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 13 Jul 2021 09:21:47 -0400 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes. -Zhengyu On 7/13/21 8:21 AM, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very > solid understanding of the AArch64 port. He has contributed 59 commits > to the main JDK repository fixing significant Arch64-related problems > across the JDK code base including the compilers, GCs, JVM core/runtime > and build system [1]. Nick has also recently taken it upon himself to > start contributing AArch64 fixes to the Valhalla project adding a > further 7 commits [2]. His wide ranging interest and understanding will > make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. > ?Votes must be cast in the open by replying to this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > From sgehwolf at redhat.com Tue Jul 13 13:22:30 2021 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 13 Jul 2021 15:22:30 +0200 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: <9e8d7722931c0752314b9ad9accf8528260b7874.camel@redhat.com> Vote: yes On Tue, 2021-07-13 at 13:21 +0100, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. From richard.reingruber at sap.com Tue Jul 13 13:39:29 2021 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Tue, 13 Jul 2021 13:39:29 +0000 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes Cheers, Richard. -----Original Message----- From: jdk-dev On Behalf Of Andrew Dinn Sent: Dienstag, 13. Juli 2021 14:22 To: jdk-dev Subject: CFV: New JDK Reviewer: Nick Gasson I hereby nominate Nick Gasson to JDK Reviewer. Nick has been contributing to the JDK project since 2018 and has a very solid understanding of the AArch64 port. He has contributed 59 commits to the main JDK repository fixing significant Arch64-related problems across the JDK code base including the compilers, GCs, JVM core/runtime and build system [1]. Nick has also recently taken it upon himself to start contributing AArch64 fixes to the Valhalla project adding a further 7 commits [2]. His wide ranging interest and understanding will make him a very valuable JDK reviewer. Votes are due by 23:00 GMT 27 July 2021. Only current JDK Reviewers [3] are eligible to vote on this nomination. Votes must be cast in the open by replying to this mailing list. For Three-Vote Consensus voting instructions, see [4]. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill [1] https://github.com/openjdk/jdk/commits?author=nick-arm [2] https://github.com/openjdk/valhalla/commits?author=nick-arm [3] https://openjdk.java.net/census [4] https://openjdk.java.net/projects/#reviewer-vote From mark.reinhold at oracle.com Thu Jul 15 21:12:34 2021 From: mark.reinhold at oracle.com (Mark Reinhold) Date: Thu, 15 Jul 2021 21:12:34 +0000 Subject: JDK 17 is now in Rampdown Phase Two Message-ID: <462AC75E-46C2-4C42-BB3C-B78F077C9E1C@oracle.com> Per the JDK 17 schedule [1], we are now in Rampdown Phase Two. The overall feature set is frozen. No further JEPs will be targeted to this release. Per the JDK Release Process [2] we now turn our focus to P1 and P2 bugs, which can be fixed with approval [3]. Late enhancements are still possible, with approval [4], but the bar is now extraordinarily high. If you?re responsible for any of the bugs on the RDP 2 candidate-bug list [5] then please see JEP 3 for guidance on how to handle them. - Mark [1] https://openjdk.java.net/projects/jdk/17/#Schedule [2] https://openjdk.java.net/jeps/3 [3] https://openjdk.java.net/jeps/3#Fix-Request-Process [4] https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process [5] https://j.mp/jdk-rdp-2 From sean.mullan at oracle.com Fri Jul 16 13:44:15 2021 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 16 Jul 2021 09:44:15 -0400 Subject: A few updates to JEP 411: Deprecate the Security Manager for Removal Message-ID: <74bc1acd-408f-6989-34cd-1c184da8329f@oracle.com> JEP 411 has been updated with a few changes: 1. The "Description" section [1] has been updated with more details on our plans for phasing out the Security Manager in Java 18 and later. 2. The "Issue warnings" section [2] has been updated with the warning messages that are emitted at startup and run time when a Security Manager has been enabled. 3. JEP 411 has been moved to the Completed state as all required tasks have been completed. Thanks, Sean [1] https://openjdk.java.net/jeps/411#Description [2] https://openjdk.java.net/jeps/411#Issue-warnings From christoph.langer at sap.com Tue Jul 20 06:23:10 2021 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 20 Jul 2021 06:23:10 +0000 Subject: GitHub/Mailing list communication defunct? Message-ID: Hi, I just recognized that all of a sudden the traffic on the OpenJDK mailing lists dropped... First I thought it's an issue with my account but looking at the pipermail archives, I don't find any mails after e.g. some time yesterday. And I guess there's still the usual activity on GitHub... Are you aware of an issue? Thanks & Best regards Christoph From david.holmes at oracle.com Tue Jul 20 06:52:44 2021 From: david.holmes at oracle.com (David Holmes) Date: Tue, 20 Jul 2021 16:52:44 +1000 Subject: GitHub/Mailing list communication defunct? In-Reply-To: References: Message-ID: Hi Christoph, Yes there is awareness of the issue. The fact this email actually got through suggests the problem has been addressed and things should start returning to normal. Cheers, David On 20/07/2021 4:23 pm, Langer, Christoph wrote: > Hi, > > I just recognized that all of a sudden the traffic on the OpenJDK mailing lists dropped... First I thought it's an issue with my account but looking at the pipermail archives, I don't find any mails after e.g. some time yesterday. And I guess there's still the usual activity on GitHub... > > Are you aware of an issue? > > Thanks & Best regards > Christoph > From christoph.langer at sap.com Tue Jul 20 06:57:30 2021 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 20 Jul 2021 06:57:30 +0000 Subject: GitHub/Mailing list communication defunct? In-Reply-To: References: Message-ID: Hi David, thanks for getting back on that. It appears to me that the mailing lists itself are working ok and you can use them via direct mail. What doesn't work is the mails originating from GitHub. So, doesn't look like things are returning to normal yet... Best regards Christoph > -----Original Message----- > From: David Holmes > Sent: Dienstag, 20. Juli 2021 08:53 > To: Langer, Christoph ; ops at openjdk.java.net; > skara-dev at openjdk.java.net; jdk-dev at openjdk.java.net > Subject: Re: GitHub/Mailing list communication defunct? > > Hi Christoph, > > Yes there is awareness of the issue. The fact this email actually got > through suggests the problem has been addressed and things should start > returning to normal. > > Cheers, > David > > On 20/07/2021 4:23 pm, Langer, Christoph wrote: > > Hi, > > > > I just recognized that all of a sudden the traffic on the OpenJDK mailing lists > dropped... First I thought it's an issue with my account but looking at the > pipermail archives, I don't find any mails after e.g. some time yesterday. And > I guess there's still the usual activity on GitHub... > > > > Are you aware of an issue? > > > > Thanks & Best regards > > Christoph > > From christoph.langer at sap.com Wed Jul 21 15:07:28 2021 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 21 Jul 2021 15:07:28 +0000 Subject: GitHub/Mailing list communication defunct? In-Reply-To: References: Message-ID: Hi, there's still no traffic on the mailing lists... It would be much appreciated if you could share an update on the status. Thanks Christoph From: Langer, Christoph Sent: Dienstag, 20. Juli 2021 08:23 To: ops at openjdk.java.net; skara-dev at openjdk.java.net; jdk-dev at openjdk.java.net Subject: GitHub/Mailing list communication defunct? Hi, I just recognized that all of a sudden the traffic on the OpenJDK mailing lists dropped... First I thought it's an issue with my account but looking at the pipermail archives, I don't find any mails after e.g. some time yesterday. And I guess there's still the usual activity on GitHub... Are you aware of an issue? Thanks & Best regards Christoph From david.holmes at oracle.com Wed Jul 21 21:29:30 2021 From: david.holmes at oracle.com (David Holmes) Date: Thu, 22 Jul 2021 07:29:30 +1000 Subject: GitHub/Mailing list communication defunct? In-Reply-To: References: Message-ID: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> Hi Christoph, On 22/07/2021 1:07 am, Langer, Christoph wrote: > Hi, > > there's still no traffic on the mailing lists... > > It would be much appreciated if you could share an update on the status. There was no update to share I'm afraid but it has now been resolved. It seems likely that the missing mail is gone forever unfortunately. Cheers, David > Thanks > Christoph > > From: Langer, Christoph > Sent: Dienstag, 20. Juli 2021 08:23 > To: ops at openjdk.java.net; skara-dev at openjdk.java.net; jdk-dev at openjdk.java.net > Subject: GitHub/Mailing list communication defunct? > > Hi, > > I just recognized that all of a sudden the traffic on the OpenJDK mailing lists dropped... First I thought it's an issue with my account but looking at the pipermail archives, I don't find any mails after e.g. some time yesterday. And I guess there's still the usual activity on GitHub... > > Are you aware of an issue? > > Thanks & Best regards > Christoph > From tim.bell at oracle.com Wed Jul 21 21:33:57 2021 From: tim.bell at oracle.com (tim.bell at oracle.com) Date: Wed, 21 Jul 2021 14:33:57 -0700 Subject: GitHub/Mailing list communication defunct? In-Reply-To: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> References: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> Message-ID: > just recognized that all of a sudden the traffic on the OpenJDK > mailing lists dropped About an hour ago we corrected a problem on the Skara server processing external (EG: github) messages.? Message traffic should be flowing again. Messages for events during the outage are lost.? We will be holding a port-mortem to investigate why this outage was not noticed sooner, and why there was a data loss. Tim On 7/21/21 14:29, David Holmes wrote: > Hi Christoph, > > On 22/07/2021 1:07 am, Langer, Christoph wrote: >> Hi, >> >> there's still no traffic on the mailing lists... >> >> It would be much appreciated if you could share an update on the status. > > There was no update to share I'm afraid but it has now been resolved. > > It seems likely that the missing mail is gone forever unfortunately. > > Cheers, > David > >> Thanks >> Christoph >> >> From: Langer, Christoph >> Sent: Dienstag, 20. Juli 2021 08:23 >> To: ops at openjdk.java.net; skara-dev at openjdk.java.net; >> jdk-dev at openjdk.java.net >> Subject: GitHub/Mailing list communication defunct? >> >> Hi, >> >> I just recognized that all of a sudden the traffic on the OpenJDK >> mailing lists dropped... First I thought it's an issue with my >> account but looking at the pipermail archives, I don't find any mails >> after e.g. some time yesterday. And I guess there's still the usual >> activity on GitHub... >> >> Are you aware of an issue? >> >> Thanks & Best regards >> Christoph >> From aph at redhat.com Thu Jul 22 08:35:42 2021 From: aph at redhat.com (Andrew Haley) Date: Thu, 22 Jul 2021 09:35:42 +0100 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: <096f0f54-eff6-f4b2-7b6d-1e034356457a@redhat.com> Vote: yes On 7/13/21 1:21 PM, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very > solid understanding of the AArch64 port. He has contributed 59 commits > to the main JDK repository fixing significant Arch64-related problems > across the JDK code base including the compilers, GCs, JVM core/runtime > and build system [1]. Nick has also recently taken it upon himself to > start contributing AArch64 fixes to the Valhalla project adding a > further 7 commits [2]. His wide ranging interest and understanding will > make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. > Votes must be cast in the open by replying to this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Thu Jul 22 10:25:11 2021 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 22 Jul 2021 12:25:11 +0200 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes On Tue, Jul 13, 2021, 14:22 Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very > solid understanding of the AArch64 port. He has contributed 59 commits > to the main JDK repository fixing significant Arch64-related problems > across the JDK code base including the compilers, GCs, JVM core/runtime > and build system [1]. Nick has also recently taken it upon himself to > start contributing AArch64 fixes to the Valhalla project adding a > further 7 commits [2]. His wide ranging interest and understanding will > make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. > Votes must be cast in the open by replying to this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > > From maurizio.cimadamore at oracle.com Thu Jul 22 10:26:05 2021 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 22 Jul 2021 11:26:05 +0100 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: <6850da25-f609-d0fd-da79-6edc9ffe2bab@oracle.com> Vote: yes! Maurizio On 13/07/2021 13:21, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a > very solid understanding of the AArch64 port. He has contributed 59 > commits to the main JDK repository fixing significant Arch64-related > problems across the JDK code base including the compilers, GCs, JVM > core/runtime and build system [1]. Nick has also recently taken it > upon himself to start contributing AArch64 fixes to the Valhalla > project adding a further 7 commits [2]. His wide ranging interest and > understanding will make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this > nomination. ?Votes must be cast in the open by replying to this > mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > From tobias.hartmann at oracle.com Thu Jul 22 11:02:48 2021 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 22 Jul 2021 13:02:48 +0200 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: <2d17e16a-b7b9-112d-045d-71228f888572@oracle.com> Vote: yes Best regards, Tobias On 13.07.21 14:21, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very solid understanding of the > AArch64 port. He has contributed 59 commits to the main JDK repository fixing significant > Arch64-related problems across the JDK code base including the compilers, GCs, JVM core/runtime and > build system [1]. Nick has also recently taken it upon himself to start contributing AArch64 fixes > to the Valhalla project adding a further 7 commits [2]. His wide ranging interest and understanding > will make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. ?Votes must be cast in the > open by replying to this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > From christoph.langer at sap.com Thu Jul 22 11:07:27 2021 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Jul 2021 11:07:27 +0000 Subject: GitHub/Mailing list communication defunct? In-Reply-To: References: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> Message-ID: Hi Tim and David, thanks for fixing this. I guess it?s not too much of an issue that events were lost. But glad that it works again. ?? Best regards Christoph From: tim.bell at oracle.com Sent: Mittwoch, 21. Juli 2021 23:34 To: David Holmes ; Langer, Christoph ; ops at openjdk.java.net; skara-dev at openjdk.java.net; jdk-dev at openjdk.java.net Subject: Re: GitHub/Mailing list communication defunct? just recognized that all of a sudden the traffic on the OpenJDK mailing lists dropped About an hour ago we corrected a problem on the Skara server processing external (EG: github) messages. Message traffic should be flowing again. Messages for events during the outage are lost. We will be holding a port-mortem to investigate why this outage was not noticed sooner, and why there was a data loss. Tim On 7/21/21 14:29, David Holmes wrote: Hi Christoph, On 22/07/2021 1:07 am, Langer, Christoph wrote: Hi, there's still no traffic on the mailing lists... It would be much appreciated if you could share an update on the status. There was no update to share I'm afraid but it has now been resolved. It seems likely that the missing mail is gone forever unfortunately. Cheers, David Thanks Christoph From: Langer, Christoph Sent: Dienstag, 20. Juli 2021 08:23 To: ops at openjdk.java.net; skara-dev at openjdk.java.net; jdk-dev at openjdk.java.net Subject: GitHub/Mailing list communication defunct? Hi, I just recognized that all of a sudden the traffic on the OpenJDK mailing lists dropped... First I thought it's an issue with my account but looking at the pipermail archives, I don't find any mails after e.g. some time yesterday. And I guess there's still the usual activity on GitHub... Are you aware of an issue? Thanks & Best regards Christoph From vladimir.kozlov at oracle.com Thu Jul 22 15:58:49 2021 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 22 Jul 2021 08:58:49 -0700 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes Thanks, Vladimir K On 7/13/21 5:21 AM, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very solid understanding of the AArch64 port. He has > contributed 59 commits to the main JDK repository fixing significant Arch64-related problems across the JDK code base > including the compilers, GCs, JVM core/runtime and build system [1]. Nick has also recently taken it upon himself to > start contributing AArch64 fixes to the Valhalla project adding a further 7 commits [2]. His wide ranging interest and > understanding will make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. ?Votes must be cast in the open by replying to > this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > From jai.forums2013 at gmail.com Thu Jul 22 16:21:27 2021 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Thu, 22 Jul 2021 21:51:27 +0530 Subject: GitHub/Mailing list communication defunct? In-Reply-To: References: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> Message-ID: <59dd2185-104f-3235-6d8a-5f581f6d99be@gmail.com> FWIW, it looks like this isn't fully back to normal. From what I can see in one of the PR's[1] I'm involved in, if I comment directly on the PR through the github UI, a mail is now getting delivered to the relevant mailing lists. So this part is back to normal now. However, if someone comments/replies directly to the mailing list thread (like Bernd and I did, during this past hour[2]), those comments aren't being registered in that github PR. This used to work previously and seems to be broken right now. [1] https://github.com/openjdk/jdk/pull/4607 [2] https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079994.html -Jaikiran On 22/07/21 3:03 am, tim.bell at oracle.com wrote: >> just recognized that all of a sudden the traffic on the OpenJDK >> mailing lists dropped > > About an hour ago we corrected a problem on the Skara server > processing external (EG: github) messages.? Message traffic should be > flowing again. > > Messages for events during the outage are lost.? We will be holding a > port-mortem to investigate why this outage was not noticed sooner, and > why there was a data loss. > > Tim > > > On 7/21/21 14:29, David Holmes wrote: >> Hi Christoph, >> >> On 22/07/2021 1:07 am, Langer, Christoph wrote: >>> Hi, >>> >>> there's still no traffic on the mailing lists... >>> >>> It would be much appreciated if you could share an update on the >>> status. >> >> There was no update to share I'm afraid but it has now been resolved. >> >> It seems likely that the missing mail is gone forever unfortunately. >> >> Cheers, >> David >> >>> Thanks >>> Christoph >>> >>> From: Langer, Christoph >>> Sent: Dienstag, 20. Juli 2021 08:23 >>> To: ops at openjdk.java.net; skara-dev at openjdk.java.net; >>> jdk-dev at openjdk.java.net >>> Subject: GitHub/Mailing list communication defunct? >>> >>> Hi, >>> >>> I just recognized that all of a sudden the traffic on the OpenJDK >>> mailing lists dropped... First I thought it's an issue with my >>> account but looking at the pipermail archives, I don't find any >>> mails after e.g. some time yesterday. And I guess there's still the >>> usual activity on GitHub... >>> >>> Are you aware of an issue? >>> >>> Thanks & Best regards >>> Christoph >>> > From vladimir.x.ivanov at oracle.com Thu Jul 22 18:54:30 2021 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 22 Jul 2021 21:54:30 +0300 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes Best regards, Vladimir Ivanov On 13.07.2021 15:21, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. From christoph.langer at sap.com Thu Jul 22 21:42:00 2021 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 22 Jul 2021 21:42:00 +0000 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote:yes /Christoph > -----Original Message----- > From: jdk-dev On Behalf Of Andrew Dinn > Sent: Dienstag, 13. Juli 2021 14:22 > To: jdk-dev > Subject: CFV: New JDK Reviewer: Nick Gasson > > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very > solid understanding of the AArch64 port. He has contributed 59 commits > to the main JDK repository fixing significant Arch64-related problems > across the JDK code base including the compilers, GCs, JVM core/runtime > and build system [1]. Nick has also recently taken it upon himself to > start contributing AArch64 fixes to the Valhalla project adding a > further 7 commits [2]. His wide ranging interest and understanding will > make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. > Votes must be cast in the open by replying to this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote From kevin.rushforth at oracle.com Thu Jul 22 22:58:46 2021 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Thu, 22 Jul 2021 15:58:46 -0700 Subject: GitHub/Mailing list communication defunct? In-Reply-To: <59dd2185-104f-3235-6d8a-5f581f6d99be@gmail.com> References: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> <59dd2185-104f-3235-6d8a-5f581f6d99be@gmail.com> Message-ID: <1ca5b885-0954-894e-f0e4-e8efc72499e6@oracle.com> Thanks for the report. This is now being tracked as a separate issue. It's being looked into, but I don't know more than that. -- Kevin On 7/22/2021 9:21 AM, Jaikiran Pai wrote: > FWIW, it looks like this isn't fully back to normal. From what I can > see in one of the PR's[1] I'm involved in, if I comment directly on > the PR through the github UI, a mail is now getting delivered to the > relevant mailing lists. So this part is back to normal now. > > However, if someone comments/replies directly to the mailing list > thread (like Bernd and I did, during this past hour[2]), those > comments aren't being registered in that github PR. This used to work > previously and seems to be broken right now. > > [1] https://github.com/openjdk/jdk/pull/4607 > > [2] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079994.html > > > -Jaikiran > > On 22/07/21 3:03 am, tim.bell at oracle.com wrote: >>> just recognized that all of a sudden the traffic on the OpenJDK >>> mailing lists dropped >> >> About an hour ago we corrected a problem on the Skara server >> processing external (EG: github) messages.? Message traffic should be >> flowing again. >> >> Messages for events during the outage are lost.? We will be holding a >> port-mortem to investigate why this outage was not noticed sooner, >> and why there was a data loss. >> >> Tim >> >> >> On 7/21/21 14:29, David Holmes wrote: >>> Hi Christoph, >>> >>> On 22/07/2021 1:07 am, Langer, Christoph wrote: >>>> Hi, >>>> >>>> there's still no traffic on the mailing lists... >>>> >>>> It would be much appreciated if you could share an update on the >>>> status. >>> >>> There was no update to share I'm afraid but it has now been resolved. >>> >>> It seems likely that the missing mail is gone forever unfortunately. >>> >>> Cheers, >>> David >>> >>>> Thanks >>>> Christoph >>>> >>>> From: Langer, Christoph >>>> Sent: Dienstag, 20. Juli 2021 08:23 >>>> To: ops at openjdk.java.net; skara-dev at openjdk.java.net; >>>> jdk-dev at openjdk.java.net >>>> Subject: GitHub/Mailing list communication defunct? >>>> >>>> Hi, >>>> >>>> I just recognized that all of a sudden the traffic on the OpenJDK >>>> mailing lists dropped... First I thought it's an issue with my >>>> account but looking at the pipermail archives, I don't find any >>>> mails after e.g. some time yesterday. And I guess there's still the >>>> usual activity on GitHub... >>>> >>>> Are you aware of an issue? >>>> >>>> Thanks & Best regards >>>> Christoph >>>> >> > From christian.hagedorn at oracle.com Fri Jul 23 07:23:00 2021 From: christian.hagedorn at oracle.com (Christian Hagedorn) Date: Fri, 23 Jul 2021 09:23:00 +0200 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes Best regards, Christian On 13.07.21 14:21, Andrew Dinn wrote: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very > solid understanding of the AArch64 port. He has contributed 59 commits > to the main JDK repository fixing significant Arch64-related problems > across the JDK code base including the compilers, GCs, JVM core/runtime > and build system [1]. Nick has also recently taken it upon himself to > start contributing AArch64 fixes to the Valhalla project adding a > further 7 commits [2]. His wide ranging interest and understanding will > make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. > Votes must be cast in the open by replying to this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > From peter.firmstone at zeus.net.au Fri Jul 23 08:36:05 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Fri, 23 Jul 2021 18:36:05 +1000 Subject: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. Message-ID: Post JEP 411, we need to write Agents to replace the current permission checks performed within the JVM by instrumenting it, following advice by OpenJDK developers, however for us this goes against all our previous development practices, no part of our codebase accesses any JDK implementation classes or internal api's. We also don't release anything that depends on deprecated API's. Modules didn't break our code, neither has tightening rules around access in JEP 403. We have been advised that we need to instrument the JDK with Agent's by OpenJDK. I am now ready to write these Agents, so that I can begin testing my new authorization layer for Java: https://github.com/pfirmstone/HighPerformanceSecurity As an example, we need to instrument java.lang.ClassLoader, in this case we need to instrument the following method: private static Void checkCreateClassLoader(String name);?? This check must be performed prior to calling java.lang.Object's constructor, to throw an exception, without making ClassLoader susceptible to a finalizer attack. Accessing private internal methods goes against all our current development practices, these are at risk of breaking in future. We cannot add methods with Agent's only change method contents. I am requesting hooks, in the form of an annotation, such as the following, so that OpenJDK developers know that this method will be instrumented by Agent's and not to change it's signature. @Instrumented private static Void checkCreateClassLoader(String name); If OpenJDK will provide instrumentation hooks, then this is a workable solution for us. Or is OpenJDK encouraging us to start accessing private methods and have to test each Java release for breakages? I'm wondering what the point of JEP 403 is if, our solution to JEP 411, is to start instrumenting private methods??? I don't think this is what OpenJDK developers are proposing. Currently removing SM will allow an attacker to cause our software running on the JVM to parse untrusted data, which previously required an authenticated client?? Permission is only granted to Principal's, of course post JEP 411, these checks will stop working in future, making our software vulnerable to attacks by unauthenticated users. We're still up in the air about how to provide credentials for our TLS and Kerberos connections, for authentication, I've created support for obtaining the Subject from the Authorization context, in my authorization layer software (linked above), but instrumenting private methods in the JDK goes against all previously learned best practices. -- Regards, Peter Firmstone Code snippet from java.lang.ClassLoader: if (name != null && name.isEmpty()) { throw new IllegalArgumentException("name must be non-empty or null"); } @SuppressWarnings("removal") SecurityManager security = System.getSecurityManager(); if (security != null) { security.checkCreateClassLoader(); } return null; } private ClassLoader(Void unused, String name, ClassLoader parent) { this.name = name; this.parent = parent; this.unnamedModule = new Module(this); if (ParallelLoaders.isRegistered(this.getClass())) { parallelLockMap = new ConcurrentHashMap<>(); assertionLock = new Object(); } else { // no finer-grained lock; lock on the classloader instance parallelLockMap = null; assertionLock = this; } this.package2certs = new ConcurrentHashMap<>(); this.nameAndId = nameAndId(this); } /** * If the defining loader has a name explicitly set then * '' @ * If the defining loader has no name then * @ * If it's built-in loader then omit `@` as there is only one instance. */ private static String nameAndId(ClassLoader ld) { String nid = ld.getName() != null ? "\'" + ld.getName() + "\'" : ld.getClass().getName(); if (!(ld instanceof BuiltinClassLoader)) { String id = Integer.toHexString(System.identityHashCode(ld)); nid = nid + " @" + id; } return nid; } /** * Creates a new class loader of the specified name and using the * specified parent class loader for delegation. * * @apiNote If the parent is specified as {@codenull} (for the * bootstrap class loader) then there is no guarantee that all platform * classes are visible. * * @param name class loader name; or {@codenull} if not named * @param parent the parent class loader * * @throws IllegalArgumentException if the given name is empty. * * @throws SecurityException * If a security manager exists and its * {@link SecurityManager#checkCreateClassLoader()} * method doesn't allow creation of a new class loader. * * @since 9 */ protected ClassLoader(String name, ClassLoader parent) { this(checkCreateClassLoader(name), name, parent); } private static Void checkCreateClassLoader(String name) { From Alan.Bateman at oracle.com Fri Jul 23 11:45:32 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 23 Jul 2021 12:45:32 +0100 Subject: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> Message-ID: <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> On 23/07/2021 11:48, Peter Firmstone wrote: > > Perhaps the solution is to replace the entire class, instead of > instrumenting one method? > > Compile a patched copy of the JVM, with modified class files, then > replace the existing classes in the JVM with the modified classes? > > Kinda like maintaining a fork, but using Agents to instrument the > original JVM with classes from the fork? > > I sure wish there was a better option, if anyone knows one, I'm all ears. JEP 411 puts the JDK on the road to dropping support for sandboxing. This means there won't be a built-in means to securely run code that has malicious intent. It means that many of the concerns for finalizer attacks go away too. In the case of the ClassLoader example in your first mail then it may be that the private static method that you want to instrument will be removed. If removed, then it should make the instrumentation a bit easier so that you can instrument the protected constructors to invokestatic your equivalent of a permission check before the invokespecial. So I think this specific case is surmountable but in general I don't think it will be tenable to patch hundreds of classes and be confident that you've got everything, esp. with a moving code base and new features. I can't tell if your "authorization layer" is for use when running with code that has malicious intent. If it is, then I don't think it will be tenable to duplicate all the deeply invasive permission checks that exist today and keep it up to date as new features and changes. When agents were muted in the early discussion on JEP 411 then the context was file and network access where several people were interested in having a means to veto access. Expanding this to have a SM equivalent be able to veto every reflective access, prevent trusted method chain and other attacks, amounts to keeping the SM forever. As regards the comments about agents having the power to instrument methods that aren't accessible to user code then that is normal. Java agents are for tools to do powerful things, they aren't intended for libraries for applications to use directly. This is why agents are opt-in on the command line. Agent maintainers weld great power and must take care to never leak the Instrumentation object to applications or libraries. -Alan From rmohta.coder at gmail.com Fri Jul 23 12:16:47 2021 From: rmohta.coder at gmail.com (Rohit Mohta) Date: Fri, 23 Jul 2021 05:16:47 -0700 Subject: JDK using gettimeofday and not clock_gettime? Message-ID: Hello, While developing an in-house tool on system time related manipulation, we learnt Java uses gettimeofday() on linux-x86_64 system call for fetching System.currentTimeMillis(). I was wondering if we thought of using clock_gettime() system call on Linux? Or if there are plans to update them to align with the POSIX recommendation. From what I can see Golang, Rust are using clock_gettime(). https://github.com/openjdk/jdk11u/blob/master/src/hotspot/os/linux/os_linux.cpp#L1295-L1300 int status = gettimeofday(&time, NULL); Per https://man7.org/linux/man-pages/man2/settimeofday.2.html POSIX.1-2008 marks *gettimeofday*() as obsolete, > recommending the use of clock_gettime(2) > instead Thanks! From fweimer at redhat.com Fri Jul 23 12:23:50 2021 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 23 Jul 2021 14:23:50 +0200 Subject: JDK using gettimeofday and not clock_gettime? In-Reply-To: (Rohit Mohta's message of "Fri, 23 Jul 2021 05:16:47 -0700") References: Message-ID: <871r7pfcbt.fsf@oldenburg.str.redhat.com> * Rohit Mohta: > While developing an in-house tool on system time related manipulation, we > learnt Java uses gettimeofday() on linux-x86_64 system call for fetching > System.currentTimeMillis(). You should not see any system call at all. The call should go to the vDSO and handled in userspace. In current glibc, the implementation of gettimeofday is based on clock_gettime (with CLOCK_REALTIME). Calling clock_gettime directly would avoid converting from nanoseconds to milliseconds in two stages, I guess, but that's the only difference. Thanks, Florian From peter.firmstone at zeus.net.au Fri Jul 23 12:35:36 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Fri, 23 Jul 2021 22:35:36 +1000 Subject: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> Message-ID: <5bf6544f-103d-a232-c6da-d16c13a25eaf@zeus.net.au> Trusted code only, but it's intended to be run in a locked down principle of least privilege environment, while we perform static analysis and targeted review, we don't have the resources required to perform thorough trusted code audits, so have been reliant on the principle of least privilege.? Hence the ability to generate policy files that we use for auditing. I utilize the constructor guarantee added in Java 6, that prevents finalizer attacks for atomic de-serialization, if invariants aren't satisfied, failure is atomic, thanks to this feature. We use the SM to ensure network connections used are secure and only have authenticated users, otherwise de-serialization and dynamic class loading is prevented. We don't need all the SM permission check features, but preventing unauthorized class loading is one of them.?? Network, file access, class loading, properties are a few that come to mind, I'm not worried about reflection, other recent changes are taking care of reflective access. We have a heap of our own Permission implementations as well. It appears that Java's future path is diverging from our software's needs and Java won't be a suitable platform in future. I wasn't aware that Java was heading in this direction.?? JEP 411 came as a big surprise, it just seemed like it was a core Java feature. I think the best course of action, will be to focus on Java 8, until that's EOL'd.?? I do like the improvements to TLS stateless connections and new features in later Java versions, but it looks like a mistake to try and keep up with Java, with this new knowledge.? I think I'll back out some more recent changes that were intended to allow our software to run on Java 17, as these cause breakages for downstream developers and it doesn't seem to make much sense now, to make those breaking changes.? I was actually looking forward to taking advantage of loom and the new vector API, but it looks like I need to be focused on what's going to be our next software platform.? I've always found llvm to be interesting, so maybe I'll focus on some languages that run on that, and implement privileged constraints at a lower level. Perhaps by the time Java 8's EOL'd there will be a new platform and language that's more suitable. Thank you for your time, I do appreciate your replies and it gives me the clarity I needed. I wish it wasn't the case, but I have to accept that times are changing. Best of luck for the future my Java friends, it's been a powerful language, especially with regard to concurrency and scaling, we got great performance, all our hotspots were native methods and Java let us get close to the metal at low levels as well using bit shift operations on primitives, that are magnitudes faster than standard string operations.?? TLS ran so much faster with stateless session tickets too. Regards, Peter. On 23/07/2021 9:45 pm, Alan Bateman wrote: > On 23/07/2021 11:48, Peter Firmstone wrote: >> >> Perhaps the solution is to replace the entire class, instead of >> instrumenting one method? >> >> Compile a patched copy of the JVM, with modified class files, then >> replace the existing classes in the JVM with the modified classes? >> >> Kinda like maintaining a fork, but using Agents to instrument the >> original JVM with classes from the fork? >> >> I sure wish there was a better option, if anyone knows one, I'm all >> ears. > > JEP 411 puts the JDK on the road to dropping support for sandboxing. > This means there won't be a built-in means to securely run code that > has malicious intent. It means that many of the concerns for finalizer > attacks go away too. In the case of the ClassLoader example in your > first mail then it may be that the private static method that you want > to instrument will be removed. If removed, then it should make the > instrumentation a bit easier so that you can instrument the protected > constructors to invokestatic your equivalent of a permission check > before the invokespecial. So I think this specific case is > surmountable but in general I don't think it will be tenable to patch > hundreds of classes and be confident that you've got everything, esp. > with a moving code base and new features. I can't tell if your > "authorization layer" is for use when running with code that has > malicious intent. If it is, then I don't think it will be tenable to > duplicate all the deeply invasive permission checks that exist today > and keep it up to date as new features and changes. When agents were > muted in the early discussion on JEP 411 then the context was file and > network access where several people were interested in having a means > to veto access. Expanding this to have a SM equivalent be able to veto > every reflective access, prevent trusted method chain and other > attacks, amounts to keeping the SM forever. > > As regards the comments about agents having the power to instrument > methods that aren't accessible to user code then that is normal. Java > agents are for tools to do powerful things, they aren't intended for > libraries for applications to use directly. This is why agents are > opt-in on the command line. Agent maintainers weld great power and > must take care to never leak the Instrumentation object to > applications or libraries. > > -Alan From david.holmes at oracle.com Fri Jul 23 13:40:21 2021 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Jul 2021 23:40:21 +1000 Subject: JDK using gettimeofday and not clock_gettime? In-Reply-To: References: Message-ID: Hi Rohit, On 23/07/2021 10:16 pm, Rohit Mohta wrote: > Hello, > > While developing an in-house tool on system time related manipulation, we > learnt Java uses gettimeofday() on linux-x86_64 system call for fetching > System.currentTimeMillis(). It doesn't in the latest release. Historically we had issues with clock_gettime being available at runtime so we used gettimeofday() when necessary. That eventually got cleaned up and we assume clock_gettime is always available on supported build and runtime platforms. I'm not sure if this was fixed for 16 or 17 ... but not backported to 11u. Cheers, David > I was wondering if we thought of using clock_gettime() system call on > Linux? Or if there are plans to update them to align with the POSIX > recommendation. From what I can see Golang, Rust are using clock_gettime(). > > https://github.com/openjdk/jdk11u/blob/master/src/hotspot/os/linux/os_linux.cpp#L1295-L1300 > int status = gettimeofday(&time, NULL); > > Per https://man7.org/linux/man-pages/man2/settimeofday.2.html > > POSIX.1-2008 marks *gettimeofday*() as obsolete, >> recommending the use of clock_gettime(2) >> instead > > > Thanks! > From david.holmes at oracle.com Fri Jul 23 13:41:57 2021 From: david.holmes at oracle.com (David Holmes) Date: Fri, 23 Jul 2021 23:41:57 +1000 Subject: GitHub/Mailing list communication defunct? In-Reply-To: <1ca5b885-0954-894e-f0e4-e8efc72499e6@oracle.com> References: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> <59dd2185-104f-3235-6d8a-5f581f6d99be@gmail.com> <1ca5b885-0954-894e-f0e4-e8efc72499e6@oracle.com> Message-ID: Hi Kevin, On 23/07/2021 8:58 am, Kevin Rushforth wrote: > Thanks for the report. This is now being tracked as a separate issue. > It's being looked into, but I don't know more than that. I'm also noticing this. Is there any update? Thanks, David > -- Kevin > > > On 7/22/2021 9:21 AM, Jaikiran Pai wrote: >> FWIW, it looks like this isn't fully back to normal. From what I can >> see in one of the PR's[1] I'm involved in, if I comment directly on >> the PR through the github UI, a mail is now getting delivered to the >> relevant mailing lists. So this part is back to normal now. >> >> However, if someone comments/replies directly to the mailing list >> thread (like Bernd and I did, during this past hour[2]), those >> comments aren't being registered in that github PR. This used to work >> previously and seems to be broken right now. >> >> [1] https://github.com/openjdk/jdk/pull/4607 >> >> [2] >> https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079994.html >> >> >> >> -Jaikiran >> >> On 22/07/21 3:03 am, tim.bell at oracle.com wrote: >>>> just recognized that all of a sudden the traffic on the OpenJDK >>>> mailing lists dropped >>> >>> About an hour ago we corrected a problem on the Skara server >>> processing external (EG: github) messages.? Message traffic should be >>> flowing again. >>> >>> Messages for events during the outage are lost.? We will be holding a >>> port-mortem to investigate why this outage was not noticed sooner, >>> and why there was a data loss. >>> >>> Tim >>> >>> >>> On 7/21/21 14:29, David Holmes wrote: >>>> Hi Christoph, >>>> >>>> On 22/07/2021 1:07 am, Langer, Christoph wrote: >>>>> Hi, >>>>> >>>>> there's still no traffic on the mailing lists... >>>>> >>>>> It would be much appreciated if you could share an update on the >>>>> status. >>>> >>>> There was no update to share I'm afraid but it has now been resolved. >>>> >>>> It seems likely that the missing mail is gone forever unfortunately. >>>> >>>> Cheers, >>>> David >>>> >>>>> Thanks >>>>> Christoph >>>>> >>>>> From: Langer, Christoph >>>>> Sent: Dienstag, 20. Juli 2021 08:23 >>>>> To: ops at openjdk.java.net; skara-dev at openjdk.java.net; >>>>> jdk-dev at openjdk.java.net >>>>> Subject: GitHub/Mailing list communication defunct? >>>>> >>>>> Hi, >>>>> >>>>> I just recognized that all of a sudden the traffic on the OpenJDK >>>>> mailing lists dropped... First I thought it's an issue with my >>>>> account but looking at the pipermail archives, I don't find any >>>>> mails after e.g. some time yesterday. And I guess there's still the >>>>> usual activity on GitHub... >>>>> >>>>> Are you aware of an issue? >>>>> >>>>> Thanks & Best regards >>>>> Christoph >>>>> >>> >> > From rmohta.coder at gmail.com Fri Jul 23 17:48:42 2021 From: rmohta.coder at gmail.com (Rohit Mohta) Date: Fri, 23 Jul 2021 10:48:42 -0700 Subject: JDK using gettimeofday and not clock_gettime? In-Reply-To: <871r7pfcbt.fsf@oldenburg.str.redhat.com> References: <871r7pfcbt.fsf@oldenburg.str.redhat.com> Message-ID: <9E46A7C8-F377-4B53-9392-0A8D90C68D77@gmail.com> Florian, Yeah it?s a vDSO call. Our tool changes the pointer to vDSO clock_get time to our function to skew time and look at system behavior from integrity standpoint. I?m running JDK 11 this on RHEL 7.8, will check the change log for the glibc version. ------ Rohit Mohta > On Jul 23, 2021, at 05:23, Florian Weimer wrote: > > ?* Rohit Mohta: > >> While developing an in-house tool on system time related manipulation, we >> learnt Java uses gettimeofday() on linux-x86_64 system call for fetching >> System.currentTimeMillis(). > > You should not see any system call at all. The call should go to the > vDSO and handled in userspace. > > In current glibc, the implementation of gettimeofday is based on > clock_gettime (with CLOCK_REALTIME). Calling clock_gettime directly > would avoid converting from nanoseconds to milliseconds in two stages, I > guess, but that's the only difference. > > Thanks, > Florian > From rmohta.coder at gmail.com Fri Jul 23 17:53:47 2021 From: rmohta.coder at gmail.com (Rohit Mohta) Date: Fri, 23 Jul 2021 10:53:47 -0700 Subject: JDK using gettimeofday and not clock_gettime? In-Reply-To: References: Message-ID: Thank you David. I see clock_gettime in JDK 17 branch in os_posix.cpp. ------ Rohit Mohta > On Jul 23, 2021, at 06:40, David Holmes wrote: > > ?Hi Rohit, > >> On 23/07/2021 10:16 pm, Rohit Mohta wrote: >> Hello, >> While developing an in-house tool on system time related manipulation, we >> learnt Java uses gettimeofday() on linux-x86_64 system call for fetching >> System.currentTimeMillis(). > > It doesn't in the latest release. Historically we had issues with clock_gettime being available at runtime so we used gettimeofday() when necessary. That eventually got cleaned up and we assume clock_gettime is always available on supported build and runtime platforms. I'm not sure if this was fixed for 16 or 17 ... but not backported to 11u. > > Cheers, > David > >> I was wondering if we thought of using clock_gettime() system call on >> Linux? Or if there are plans to update them to align with the POSIX >> recommendation. From what I can see Golang, Rust are using clock_gettime(). >> https://github.com/openjdk/jdk11u/blob/master/src/hotspot/os/linux/os_linux.cpp#L1295-L1300 >> int status = gettimeofday(&time, NULL); >> Per https://man7.org/linux/man-pages/man2/settimeofday.2.html >> POSIX.1-2008 marks *gettimeofday*() as obsolete, >>> recommending the use of clock_gettime(2) >>> instead >> Thanks! From peter.firmstone at zeus.net.au Fri Jul 23 22:33:25 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sat, 24 Jul 2021 08:33:25 +1000 Subject: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> Message-ID: <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> I think it's worth noting that there isn't a way to securely run code with malicious intent now, so I'm surprised that at this late stage you were still providing support for sand boxing (whack a mole). It's just for us many assumptions have been made on a Java platform with SM, using POLP (not sandboxing) as this was one of the foundational principles of secure coding guidelines (just like following concurrency best practice, were were following security best practice).?? Sandboxing is an all or nothing approach, if you had a trusted applet that was signed, it had AllPermission, if you had an unsigned applet, then it had no permissions.? Sandboxing was one of the use cases for SM, when combined with ClassLoader visibility, but we never realized that OpenJDK developers meant sandboxing == authorization access controls. When you remove that pillar, everything it's supporting collapses, not just sand boxing, so when you say you are removing support for sandboxing, we say, good idea, but we didn't realize you were saying you were removing support for all authorization access controls.?? Reduced and revised authorization and access control would have been acceptable, as tightening reflection visibility using a different form of access control removes the need for authorization based reflection access checks, but also removing atomic construction guarantee's just seems like were doing this at a rapid pace without the community understanding what you have in mind, and this may have more uses than just stopping finalizer attacks.? Unfortunately when you develop a feature, you can't be sure developers won't adapt and utilize it for multiple purposes. Personally I think a better approach would have been to first reduce and simplify authorization access controls, replacing some of the functionality with different but more appropriate mechanisms. Removing authorization access control features, without replacement means our software would be insecure, there isn't an obvious way to re-secure it, without re-architecting or re-designing it from the ground up, and Java is now a moving target anyway, so we would have to wait for it to re-stabilize as it transitions from Hippy to Hipster Java (just making a point it's not the same Java). With this new understanding, we need to reconsider both the language and the platform that we'll be developing on.? Clearly a language with less boilerplate is an obvious start, I don't yet know which, we will still consider the JVM, but it would be with Kafka or Clojure, but then we also need to consider whether we will be able to secure the underlying platform, or at least use it securely.? Arguably we can do things now that aren't possible on other platforms, so we need to develop that capability as well, not just secure it. Regards, Peter. On 23/07/2021 9:45 pm, Alan Bateman wrote: > On 23/07/2021 11:48, Peter Firmstone wrote: >> >> Perhaps the solution is to replace the entire class, instead of >> instrumenting one method? >> >> Compile a patched copy of the JVM, with modified class files, then >> replace the existing classes in the JVM with the modified classes? >> >> Kinda like maintaining a fork, but using Agents to instrument the >> original JVM with classes from the fork? >> >> I sure wish there was a better option, if anyone knows one, I'm all >> ears. > > JEP 411 puts the JDK on the road to dropping support for sandboxing. > This means there won't be a built-in means to securely run code that > has malicious intent. It means that many of the concerns for finalizer > attacks go away too. In the case of the ClassLoader example in your > first mail then it may be that the private static method that you want > to instrument will be removed. If removed, then it should make the > instrumentation a bit easier so that you can instrument the protected > constructors to invokestatic your equivalent of a permission check > before the invokespecial. So I think this specific case is > surmountable but in general I don't think it will be tenable to patch > hundreds of classes and be confident that you've got everything, esp. > with a moving code base and new features. I can't tell if your > "authorization layer" is for use when running with code that has > malicious intent. If it is, then I don't think it will be tenable to > duplicate all the deeply invasive permission checks that exist today > and keep it up to date as new features and changes. When agents were > muted in the early discussion on JEP 411 then the context was file and > network access where several people were interested in having a means > to veto access. Expanding this to have a SM equivalent be able to veto > every reflective access, prevent trusted method chain and other > attacks, amounts to keeping the SM forever. > > As regards the comments about agents having the power to instrument > methods that aren't accessible to user code then that is normal. Java > agents are for tools to do powerful things, they aren't intended for > libraries for applications to use directly. This is why agents are > opt-in on the command line. Agent maintainers weld great power and > must take care to never leak the Instrumentation object to > applications or libraries. > > -Alan From peter.firmstone at zeus.net.au Sat Jul 24 23:30:09 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sun, 25 Jul 2021 09:30:09 +1000 Subject: JEP 411: Sandboxing v's LOTJ (language other than Java) running on top of Java class libraries / JVM. Message-ID: <59b43c8f-62d0-c519-02c1-c951be124442@zeus.net.au> I have established it's not practical to implement agents to intercept Java class libraries (Not the JVM) to guard access, such as class loading, properties, IO, etc. It's also not practical to construct a Sandbox using ClassLoader, as has been suggested: 1. We would have to prevent access to java.lang.Class, to prevent code escaping the sandbox, this is far too restrictive. 2. It isn't practical to dynamically grant access, from within a sandbox. 3. The sandbox is an all or nothing approach. 4. The sandbox isn't an authorization layer. Clearly Java in future, will be a zone of implicit trust, similarly to how we use C today from Java. We need a "safer" language than Java, just like Java was a "safer" language than C. This "safer" language would then allow authorization access controls, for network, file, class loading, data parsing, etc. Rather than a sandbox, it just needs to be safer, with the ability to allow access to the underlying Java / C / etc, to trusted / authenticated / verified entities. If anyone has any ideas regarding suitable languages, I'd be very interested to hear your thoughts. -- Regards, Peter Firmstone From Alan.Bateman at oracle.com Sun Jul 25 14:44:19 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 25 Jul 2021 15:44:19 +0100 Subject: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> Message-ID: <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> On 23/07/2021 23:33, Peter Firmstone wrote: > I think it's worth noting that there isn't a way to securely run code > with malicious intent now, so I'm surprised that at this late stage > you were still providing support for sand boxing (whack a mole). > > It's just for us many assumptions have been made on a Java platform > with SM, using POLP (not sandboxing) as this was one of the > foundational principles of secure coding guidelines (just like > following concurrency best practice, were were following security best > practice).?? Sandboxing is an all or nothing approach, if you had a > trusted applet that was signed, it had AllPermission, if you had an > unsigned applet, then it had no permissions.? Sandboxing was one of > the use cases for SM, when combined with ClassLoader visibility, but > we never realized that OpenJDK developers meant sandboxing == > authorization access controls. > > When you remove that pillar, everything it's supporting collapses, not > just sand boxing, so when you say you are removing support for > sandboxing, we say, good idea, but we didn't realize you were saying > you were removing support for all authorization access controls.?? > Reduced and revised authorization and access control would have been > acceptable, as tightening reflection visibility using a different form > of access control removes the need for authorization based reflection > access checks, but also removing atomic construction guarantee's just > seems like were doing this at a rapid pace without the community > understanding what you have in mind, and this may have more uses than > just stopping finalizer attacks. I'm not 100% sure what you mean by "atomic construction guarantee" here. This JEP does not propose to change anything with finalization or do anything with the registration of finalizers after Object. runs. Our exchange in the previous mails was about classes (using ClassLoader as the example) that specify a SM permission check in their constructors, something that is strongly discouraged as the checks are easy to bypass. The idiom that we use in the JDK to prevent bypassing these SM permission checks with a finalizer attack is to check in a static method that returns a dummy parameter for the invokespecial. My point in the previous mail is that when the SM permission checks eventually go away then many of the uses of this idiom can do away too. That said, there is strong desire to eventually remove finalization too. Finalization was deprecated several years ago and the Java platform defines APIs that provide much more flexible and efficient ways do run cleanup actions when an object becomes unreachable. So another multi-year/multi-release effort to remove a problematic feature, just nothing to do with this JEP. As regards POLP, the focus of SM architecture when it was enhanced in Java 1.2. The JEP attempts to explain why this has been a failure. AccessController is voodoo that most developers have never encountered so anyone trying to run with SM ends up running counter to the principle of least privilege by granting all permissions. -Alan. From Alan.Bateman at oracle.com Sun Jul 25 14:48:31 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 25 Jul 2021 15:48:31 +0100 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: <431b70bb-21e8-196e-28c9-7ff9259638c4@oracle.com> Vote: yes From peter.firmstone at zeus.net.au Mon Jul 26 01:05:32 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Mon, 26 Jul 2021 11:05:32 +1000 Subject: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> Message-ID: <7ed9b7ec-85dd-e804-90c6-140c1c7ee3fa@zeus.net.au> Hi Alan, My apologies, finalizer removal, that's a positive step, I thought it was motivated by "the code is trusted so we don't need to worry about finalizer attacks". Also apologies for the long email, but it seems appropriate for information handover. Atomic construction guarantee; if invariant checks during construction throw an exception, prior to calling Object's zero arg constructor, an object instance will not be created, creation of the object is atomic, either it succeeds with all invariants satisfied, or an object is not created, such that it cannot be created in a broken state where invariants aren't satisfied.? We apply this constructor safety guarantee to deserialization of data, it is a requirement of the code performing deserialzation to check invariants and we require that users are authenticated, this is to avoid parsing untrusted data, while still validating user data.? We enforce it using SM infrastructure. I understand JEP 411 is a business decision, there wasn't enough adoption, following the fallout of applets, businesses and users running afoul of untrusted code, causing ongoing pain (publicly). The remaining attempts of JEP 411 to explain why POLP is a technical failure only apply to the default implementation and are incorrect when applied to other implementations, it's a commercial failure, as suggested by low rates of adoption in JEP 411, but that's due to a lack of investment in tooling, however I suspect OpenJDK has underestimated adoption, although probably not by a big margin, but I suspect it will be more painful than OpenJDK anticipates.? I have a perfectly good working reliable publicly available example (for years) contrary to JEP 411's technical claims. OpenJDK's decision has been made, and those affected must also assess and make their own decisions, the following only serves to share my thoughts and insights, no need to read further if not of interest.? Our Java programs are going into care and maintenance as we assess suitable replacement development platforms. <---------> Applets relied on SM (perhaps SM only exists due to their success), applets themselves weren't the cause of their own demise, for that we have Java Serialization to thank, otherwise applets were a commercial success, and had they remained so, then SM would have also remained, it appears to be inexorably tied to the fate of applets now. Serialization needed an atomic replacement before 2008, when it was becoming obvious that Java serialization was insecure. OpenJDK could still fix Java serialization without using white list filters (ironically white listing is a complication of SM, which reduced adoption, it's likely the same will occur with Serialization white lists, if tooling isn't provided), by removing the ability to serialize circular object graphs, or disabling it by default.? We had circular object graphs in JGDMS (which heavily utilised Java serialization), but we refactored these out after implementing atomic de-serialization, we did this in a way that didn't require breaking the serial form compatibility of existing classes (unless they contained circular references).? This keeps serial data invariant validation code with the object implementation, rather than as a separate white list (and it is more powerful and less complex than white listing), reducing complexity and maintenance, and because failure is atomic an attacker cannot formulate a gadget chain, type safety is also read ahead and checked prior to de-serialization of data.?? The development of atomic serialization started with atomic deserialization which was completed a few years ago, atomic serialization was under current development, with new explicit public API methods for used for serialization, to avoid any issues with reflection and module access controls, we were still using Java serialization to serialize, but an alternative AtomicObjectInputStream to de-serialize. SM Performance isn't an issue, my policy implementation is high scaling and has no hotspots, neither is deployment, we have tools to generate policy files (more than one) and have been doing so for many years (the first tool was written by Sun Microsystems circa 2004 I think, it still required policy file editing, but listed permissions required), the second tool was written 8 years ago approx.? Our use cases have passed the tests of time.? I don't believe people hand author policy files in the age of computing: I've seen examples of policy generation tools by other authors on GitHub.? Sure some developers might grant AllPermission, to get something running, or for tests, but I haven't seen anyone serious about security in production that does.? I don't use the built in policy provider (it has a blocking permission cache that negatively impacts performance), my policy implementation doesn't have a cache and it's many magnitudes faster and high scaling thanks to shared immutability, thread confined mutability and the garbage collector, the last remaining pain point is SocketPermssion and DNS.? If SocketPermission was changed to use URI RFC3986 normalization and had netmask wildcards, that would address remaining issues (Too many SocketPermission grants required in policy files).?? I managed to avoid most DNS calls by using a PermissionComparator, which finds the closest match, if it doesn't find an exact match, to reduce the number of actual permission checks.?? We also had to implement our own ClassLoader to avoid the DNS calls originating from SecureClassLoader. Java's memory model has more voodoo than AccessController, the JMM makes AccessController look simple, programming is voodoo to a non programmer, everything is relative and based on experience.? I guess you are suggesting the complexity to use ratio is too high. I can imagine what it was like being at a presentation when someone enables SecurityManager and it stops working, why the default provider wasn't fixed then I'm not sure, budget perhaps? If I was at that presentation, I'd generate the policy file on the first pass, then make the new policy file the default policy.? It would have added about 10 minutes to the presentation as we stepped through the swim lanes, or whatever people like to call executing desired functionality (preferably automated test cases), at least it would look like we knew what we were doing, then you validate it by attempting to do something you shouldn't, it leaves the JVM in a very locked down state (which can also prevent user errors), occasionally you can forget to execute some necessary functionality (not have a test case for it), or not allow the program to run long enough to capture all necessary permissions, but the missing permissions are quickly sorted, by running the tool a second time, the missing permissions are appended to policy files. I would have preferred that we reduced the number of permissions, these can be removed without breaking backward compatibility, the policy just treats it as an UnresolvedPermission. Many permissions are being replaced by better access controls (as they should), such as more recent changes to package / module access and reflection and this could be a gradual process as unnecessary permissions are removed.?? We aren't trying to sandbox code, it's used for company authorization rules.? The code is trusted, but if it's from a third party, or another company with a business relationship, eg approved vendor, we need to place some constraints on it, which aren't intended to defend against malicious code.? Our intent is to prevent parsing of malicious data and loading of malicious code. I would have preferred to build a model around permissions required for authorization of users and trusted code, rather than focusing on sandboxes and malicious code.?? If a user or service doesn't authenticate, they cannot dynamically load code because they are not granted permission to do so, they also cannot generally deserialize untrusted data, this all depends on SM infrastructure. We need to control access to networks, files, user credentials, properties (of certificate store locations) and class loading. Primarily we do this by authenticating users, we also allow some authenticated users and services to also dynamically download and load classes. Due to JEP 411, we have no future Java upgrade path, what's possible currently with authorization, will not be possible in future versions of Java, design assumptions throughout our software are built on SM infrastructure.?? When we remove SM from our systems, it enables deserialization of untrusted data from unauthenticated users and it allows downloads and class loading of untrusted code.?? Our software uses TLS over IPv6 for it's point to point connectivity and is able to dynamically discover compatible services over global networks, so we cannot use a firewall to protect it either. The library we use and provide publicy can be found here, in case its of interest: https://github.com/pfirmstone/JGDMS At the end of the day, choosing any development platform is a risk, and this was one of the risks of choosing a commercial platform (at that time Java was a commercial platform and it's still funded by commercial interests today), budgets dictate both the initial development compromises and the eventual demise when adoption falls and a feature is no longer commercially viable for the people responsible for maintaining it. Due to JEP 411 all our Java development will be put into care and maintenance, I'm currently looking at our options for other programming languages and platforms on which to base new development, Haskell looks interesting and seems to have better type safety, due to it's academic background, its developers are focused on solving very difficult problems and doing so in a way that is provably correct, using mathematical theorems, I think that's why its taken years longer to stabilize.? By not having made compromises, it will likely be useful for much longer, even with some change.? It seems unlikely that an academic language would lose features due to budget constraints, it is more likely that inadequate or problematic features will be addressed and upgraded or replaced. It is not so much that JEP 411 might break backward compatibility, we can live with that, what we are unable to address; it removes a feature that cannot be re-implemented and has no replacement, which exposes us to low probability, but unacceptable consequences. There are no hard feelings, it's just a consequence of our original platform adoption choice, we knew there were risks.? It's time to move on and deal with it.? No doubt Java will be useful to many people for many years to come, and many don't require an authorization layer or chose something other than SM to implement one.? With no Java upgrade path, it leaves us free to choose from what is available now, rather than a choice made 20 years ago. In any case it's likely a choice that we would have needed to make eventually, JEP 411 has only brought it forward.?? If Haskell is a magnitude more efficient, as its proponents claim, then it may ultimately provide an overall cost saving.?? We haven't made a choice yet though, it's still under investigation. I do appreciate that you took the time to respond to my emails. Regards, Peter. On 26/07/2021 12:44 am, Alan Bateman wrote: > On 23/07/2021 23:33, Peter Firmstone wrote: >> I think it's worth noting that there isn't a way to securely run code >> with malicious intent now, so I'm surprised that at this late stage >> you were still providing support for sand boxing (whack a mole). >> >> It's just for us many assumptions have been made on a Java platform >> with SM, using POLP (not sandboxing) as this was one of the >> foundational principles of secure coding guidelines (just like >> following concurrency best practice, were were following security >> best practice).?? Sandboxing is an all or nothing approach, if you >> had a trusted applet that was signed, it had AllPermission, if you >> had an unsigned applet, then it had no permissions.? Sandboxing was >> one of the use cases for SM, when combined with ClassLoader >> visibility, but we never realized that OpenJDK developers meant >> sandboxing == authorization access controls. >> >> When you remove that pillar, everything it's supporting collapses, >> not just sand boxing, so when you say you are removing support for >> sandboxing, we say, good idea, but we didn't realize you were saying >> you were removing support for all authorization access controls.?? >> Reduced and revised authorization and access control would have been >> acceptable, as tightening reflection visibility using a different >> form of access control removes the need for authorization based >> reflection access checks, but also removing atomic construction >> guarantee's just seems like were doing this at a rapid pace without >> the community understanding what you have in mind, and this may have >> more uses than just stopping finalizer attacks. > I'm not 100% sure what you mean by "atomic construction guarantee" > here. This JEP does not propose to change anything with finalization > or do anything with the registration of finalizers after Object. > runs. Our exchange in the previous mails was about classes (using > ClassLoader as the example) that specify a SM permission check in > their constructors, something that is strongly discouraged as the > checks are easy to bypass. The idiom that we use in the JDK to prevent > bypassing these SM permission checks with a finalizer attack is to > check in a static method that returns a dummy parameter for the > invokespecial. My point in the previous mail is that when the SM > permission checks eventually go away then many of the uses of this > idiom can do away too. > > That said, there is strong desire to eventually remove finalization > too. Finalization was deprecated several years ago and the Java > platform defines APIs that provide much more flexible and efficient > ways do run cleanup actions when an object becomes unreachable. So > another multi-year/multi-release effort to remove a problematic > feature, just nothing to do with this JEP. > > As regards POLP, the focus of SM architecture when it was enhanced in > Java 1.2. The JEP attempts to explain why this has been a failure. > AccessController is voodoo that most developers have never encountered > so anyone trying to run with SM ends up running counter to the > principle of least privilege by granting all permissions. > > -Alan. From volker.simonis at gmail.com Mon Jul 26 17:14:45 2021 From: volker.simonis at gmail.com (Volker Simonis) Date: Mon, 26 Jul 2021 19:14:45 +0200 Subject: CFV: New JDK Reviewer: Nick Gasson In-Reply-To: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> References: <5e8cb533-d4a7-7273-229d-53c6892b2cc9@redhat.com> Message-ID: Vote: yes Andrew Dinn schrieb am Di., 13. Juli 2021, 14:22: > I hereby nominate Nick Gasson to JDK Reviewer. > > Nick has been contributing to the JDK project since 2018 and has a very > solid understanding of the AArch64 port. He has contributed 59 commits > to the main JDK repository fixing significant Arch64-related problems > across the JDK code base including the compilers, GCs, JVM core/runtime > and build system [1]. Nick has also recently taken it upon himself to > start contributing AArch64 fixes to the Valhalla project adding a > further 7 commits [2]. His wide ranging interest and understanding will > make him a very valuable JDK reviewer. > > Votes are due by 23:00 GMT 27 July 2021. > > Only current JDK Reviewers [3] are eligible to vote on this nomination. > Votes must be cast in the open by replying to this mailing list. > > For Three-Vote Consensus voting instructions, see [4]. > > regards, > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > > [1] https://github.com/openjdk/jdk/commits?author=nick-arm > [2] https://github.com/openjdk/valhalla/commits?author=nick-arm > > [3] https://openjdk.java.net/census > [4] https://openjdk.java.net/projects/#reviewer-vote > > From Rony.Flatscher at wu.ac.at Tue Jul 27 10:01:30 2021 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 27 Jul 2021 12:01:30 +0200 Subject: Ad removing finalize eventually (Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> Message-ID: <67886c79-aef1-bc41-0057-9acb943a0e0f@wu.ac.at> Ad Alan's remark about eventually removing finalize in this thread: On 25.07.2021 16:44, Alan Bateman wrote: ... cut ... > > That said, there is strong desire to eventually remove finalization too. Finalization was > deprecated several years ago and the Java platform defines APIs that provide much more flexible > and efficient ways do run cleanup actions when an object becomes unreachable. So another > multi-year/multi-release effort to remove a problematic feature, just nothing to do with this JEP. ... cut ... A question: what alternatives are you thinking of? Background of the question: in an ooRexx-Java bridge for each Java object an ooRexx peer object gets created and needs to exist as long as the Java object exists. Once the Java object gets garbage collected the ooRexx peer object needs to be freed as well. The bridge between ooRexx (an interpreted, message based, dynamically typed language) and Java is realized via JNI. Currently (actually since 20 years) on the Java side the finalize method gets employed to signal to ooRexx that its ooRexx peer object needs to be removed. So when you think of removing finalize and think of replacements for it, is there already a replacement devised which can be used for determining that a Java "object became unreachable"/is not referred to anymore/about to be garbage collected (or not being interacted with anymore for good) and be able to communicate that fact/event? What would that be? Are there already discussions, ideas about this somewhere? ---rony P.S.: I remember the comment back then that finalize although deprecated for use (not for removal) for Java programs will not be removed as such use cases as described above need to use it. From peter.firmstone at zeus.net.au Tue Jul 27 11:24:01 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Tue, 27 Jul 2021 21:24:01 +1000 Subject: Ad removing finalize eventually (Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <67886c79-aef1-bc41-0057-9acb943a0e0f@wu.ac.at> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> <67886c79-aef1-bc41-0057-9acb943a0e0f@wu.ac.at> Message-ID: <6415ef59-cd75-9883-67f8-651e6be0ce25@zeus.net.au> PhantomReference like this: https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-collections/src/main/java/org/apache/river/concurrent/ReferenceProcessor.java On 27/07/2021 8:01 pm, Rony G. Flatscher wrote: > Ad Alan's remark about eventually removing finalize in this thread: > > On 25.07.2021 16:44, Alan Bateman wrote: > > ... cut ... >> That said, there is strong desire to eventually remove finalization too. Finalization was >> deprecated several years ago and the Java platform defines APIs that provide much more flexible >> and efficient ways do run cleanup actions when an object becomes unreachable. So another >> multi-year/multi-release effort to remove a problematic feature, just nothing to do with this JEP. > ... cut ... > > A question: what alternatives are you thinking of? > > Background of the question: in an ooRexx-Java bridge for each Java object an ooRexx peer object gets > created and needs to exist as long as the Java object exists. Once the Java object gets garbage > collected the ooRexx peer object needs to be freed as well. The bridge between ooRexx (an > interpreted, message based, dynamically typed language) and Java is realized via JNI. Currently > (actually since 20 years) on the Java side the finalize method gets employed to signal to ooRexx > that its ooRexx peer object needs to be removed. > > So when you think of removing finalize and think of replacements for it, is there already a > replacement devised which can be used for determining that a Java "object became unreachable"/is not > referred to anymore/about to be garbage collected (or not being interacted with anymore for good) > and be able to communicate that fact/event? What would that be? Are there already discussions, ideas > about this somewhere? > > ---rony > > P.S.: I remember the comment back then that finalize although deprecated for use (not for removal) > for Java programs will not be removed as such use cases as described above need to use it. > > From Rony.Flatscher at wu.ac.at Tue Jul 27 13:42:27 2021 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 27 Jul 2021 15:42:27 +0200 Subject: Ad removing finalize eventually (Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <6415ef59-cd75-9883-67f8-651e6be0ce25@zeus.net.au> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> <67886c79-aef1-bc41-0057-9acb943a0e0f@wu.ac.at> <6415ef59-cd75-9883-67f8-651e6be0ce25@zeus.net.au> Message-ID: <49fcd0d9-1b9a-6536-1b05-a4eca4a83be6@wu.ac.at> Peter, On 27.07.2021 13:24, Peter Firmstone wrote: > PhantomReference > > like this: > > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-collections/src/main/java/org/apache/river/concurrent/ReferenceProcessor.java > thank you very much for the pointer! ---rony > > On 27/07/2021 8:01 pm, Rony G. Flatscher wrote: >> Ad Alan's remark about eventually removing finalize in this thread: >> >> On 25.07.2021 16:44, Alan Bateman wrote: >> >> ... cut ... >>> That said, there is strong desire to eventually remove finalization too. Finalization was >>> deprecated several years ago and the Java platform defines APIs that provide much more flexible >>> and efficient ways do run cleanup actions when an object becomes unreachable. So another >>> multi-year/multi-release effort to remove a problematic feature, just nothing to do with this JEP. >> ... cut ... >> >> A question: what alternatives are you thinking of? >> >> Background of the question: in an ooRexx-Java bridge for each Java object an ooRexx peer object gets >> created and needs to exist as long as the Java object exists. Once the Java object gets garbage >> collected the ooRexx peer object needs to be freed as well. The bridge between ooRexx (an >> interpreted, message based, dynamically typed language) and Java is realized via JNI. Currently >> (actually since 20 years) on the Java side the finalize method gets employed to signal to ooRexx >> that its ooRexx peer object needs to be removed. >> >> So when you think of removing finalize and think of replacements for it, is there already a >> replacement devised which can be used for determining that a Java "object became unreachable"/is not >> referred to anymore/about to be garbage collected (or not being interacted with anymore for good) >> and be able to communicate that fact/event? What would that be? Are there already discussions, ideas >> about this somewhere? >> >> ---rony >> >> P.S.: I remember the comment back then that finalize although deprecated for use (not for removal) >> for Java programs will not be removed as such use cases as described above need to use it. From roger.riggs at oracle.com Tue Jul 27 13:49:44 2021 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 27 Jul 2021 09:49:44 -0400 Subject: Ad removing finalize eventually (Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <49fcd0d9-1b9a-6536-1b05-a4eca4a83be6@wu.ac.at> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> <67886c79-aef1-bc41-0057-9acb943a0e0f@wu.ac.at> <6415ef59-cd75-9883-67f8-651e6be0ce25@zeus.net.au> <49fcd0d9-1b9a-6536-1b05-a4eca4a83be6@wu.ac.at> Message-ID: <12d2a50d-49e6-a1e9-8e4b-4c939d7777d6@oracle.com> Hi Rony, Also take a look at the java.lang.ref.Cleaner [1] added to in JDK 9. It is designed as a replacement for finalization; it uses the PhantomReferences to register cleanup actions. Regards, Roger [1] https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/ref/Cleaner.html On 7/27/21 9:42 AM, Rony G. Flatscher wrote: > Peter, > > On 27.07.2021 13:24, Peter Firmstone wrote: >> PhantomReference >> >> like this: >> >> https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-collections/src/main/java/org/apache/river/concurrent/ReferenceProcessor.java >> > thank you very much for the pointer! > > ---rony > >> On 27/07/2021 8:01 pm, Rony G. Flatscher wrote: >>> Ad Alan's remark about eventually removing finalize in this thread: >>> >>> On 25.07.2021 16:44, Alan Bateman wrote: >>> >>> ... cut ... >>>> That said, there is strong desire to eventually remove finalization too. Finalization was >>>> deprecated several years ago and the Java platform defines APIs that provide much more flexible >>>> and efficient ways do run cleanup actions when an object becomes unreachable. So another >>>> multi-year/multi-release effort to remove a problematic feature, just nothing to do with this JEP. >>> ... cut ... >>> >>> A question: what alternatives are you thinking of? >>> >>> Background of the question: in an ooRexx-Java bridge for each Java object an ooRexx peer object gets >>> created and needs to exist as long as the Java object exists. Once the Java object gets garbage >>> collected the ooRexx peer object needs to be freed as well. The bridge between ooRexx (an >>> interpreted, message based, dynamically typed language) and Java is realized via JNI. Currently >>> (actually since 20 years) on the Java side the finalize method gets employed to signal to ooRexx >>> that its ooRexx peer object needs to be removed. >>> >>> So when you think of removing finalize and think of replacements for it, is there already a >>> replacement devised which can be used for determining that a Java "object became unreachable"/is not >>> referred to anymore/about to be garbage collected (or not being interacted with anymore for good) >>> and be able to communicate that fact/event? What would that be? Are there already discussions, ideas >>> about this somewhere? >>> >>> ---rony >>> >>> P.S.: I remember the comment back then that finalize although deprecated for use (not for removal) >>> for Java programs will not be removed as such use cases as described above need to use it. From Rony.Flatscher at wu.ac.at Tue Jul 27 16:23:05 2021 From: Rony.Flatscher at wu.ac.at (Rony G. Flatscher) Date: Tue, 27 Jul 2021 18:23:05 +0200 Subject: Ad removing finalize eventually (Re: JEP 411 Headaches: Instrumenting private methods in the JDK for authorization checkpoints. In-Reply-To: <12d2a50d-49e6-a1e9-8e4b-4c939d7777d6@oracle.com> References: <19858490-648a-d539-2429-b1999f597658@zeus.net.au> <8d6f394d-f0a8-fa41-24ec-f8437af99d4f@oracle.com> <8f9e9e58-d5bb-0a99-7f3e-937a33c9e2b8@zeus.net.au> <0ae93ef1-3416-9cc0-d6e3-2fe1ccb1df4f@oracle.com> <67886c79-aef1-bc41-0057-9acb943a0e0f@wu.ac.at> <6415ef59-cd75-9883-67f8-651e6be0ce25@zeus.net.au> <49fcd0d9-1b9a-6536-1b05-a4eca4a83be6@wu.ac.at> <12d2a50d-49e6-a1e9-8e4b-4c939d7777d6@oracle.com> Message-ID: <68618c16-6697-192a-3135-b6b81ccb36a8@wu.ac.at> Hi Roger, On 27.07.2021 15:49, Roger Riggs wrote: > Also take a look at the java.lang.ref.Cleaner > [1] added > to in JDK 9. > It is designed as a replacement for finalization; it uses the PhantomReferences to register > cleanup actions. thank you very much for this pointer too! Currently, I have a need to keep the ooRexx-Java bridge at Java 1.6 as the baseline (which will increase over time to Java 9, but this may very well take years judging how "overlong" some companies keep running fully amortized systems). [The bridge still takes advantage of newer Java versions, if necessary, e.g. the needed runtime reflection is carried out with java.lang.reflect, starting with Java 8 with java.lang.invoke instead.] ---rony > > On 7/27/21 9:42 AM, Rony G. Flatscher wrote: >> Peter, >> >> On 27.07.2021 13:24, Peter Firmstone wrote: >>> PhantomReference >>> >>> like this: >>> >>> https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-collections/src/main/java/org/apache/river/concurrent/ReferenceProcessor.java >>> >>> >> thank you very much for the pointer! >> >> ---rony >> >>> On 27/07/2021 8:01 pm, Rony G. Flatscher wrote: >>>> Ad Alan's remark about eventually removing finalize in this thread: >>>> >>>> On 25.07.2021 16:44, Alan Bateman wrote: >>>> >>>> ... cut ... >>>>> That said, there is strong desire to eventually remove finalization too. Finalization was >>>>> deprecated several years ago and the Java platform defines APIs that provide much more flexible >>>>> and efficient ways do run cleanup actions when an object becomes unreachable. So another >>>>> multi-year/multi-release effort to remove a problematic feature, just nothing to do with this >>>>> JEP. >>>> ... cut ... >>>> >>>> A question: what alternatives are you thinking of? >>>> >>>> Background of the question: in an ooRexx-Java bridge for each Java object an ooRexx peer object >>>> gets >>>> created and needs to exist as long as the Java object exists. Once the Java object gets garbage >>>> collected the ooRexx peer object needs to be freed as well. The bridge between ooRexx (an >>>> interpreted, message based, dynamically typed language) and Java is realized via JNI. Currently >>>> (actually since 20 years) on the Java side the finalize method gets employed to signal to ooRexx >>>> that its ooRexx peer object needs to be removed. >>>> >>>> So when you think of removing finalize and think of replacements for it, is there already a >>>> replacement devised which can be used for determining that a Java "object became >>>> unreachable"/is not >>>> referred to anymore/about to be garbage collected (or not being interacted with anymore for good) >>>> and be able to communicate that fact/event? What would that be? Are there already discussions, >>>> ideas >>>> about this somewhere? >>>> >>>> ---rony >>>> >>>> P.S.: I remember the comment back then that finalize although deprecated for use (not for removal) >>>> for Java programs will not be removed as such use cases as described above need to use it. From tim.bell at oracle.com Tue Jul 27 17:44:16 2021 From: tim.bell at oracle.com (tim.bell at oracle.com) Date: Tue, 27 Jul 2021 10:44:16 -0700 Subject: [External] : Re: GitHub/Mailing list communication defunct? In-Reply-To: <59dd2185-104f-3235-6d8a-5f581f6d99be@gmail.com> References: <095fc050-dbb1-107d-eac9-89c974ba42b8@oracle.com> <59dd2185-104f-3235-6d8a-5f581f6d99be@gmail.com> Message-ID: Hello > if someone comments/replies directly to the mailing list thread (like > Bernd and I did, during this past hour[2]), those comments aren't > being registered in that github PR. This used to work previously and > seems to be broken right now. We are aware of this problem and still investigating.? We don't have a root cause or an ETA for this. Tim On 7/22/21 09:21, Jaikiran Pai wrote: > FWIW, it looks like this isn't fully back to normal. From what I can > see in one of the PR's[1] I'm involved in, if I comment directly on > the PR through the github UI, a mail is now getting delivered to the > relevant mailing lists. So this part is back to normal now. > > However, if someone comments/replies directly to the mailing list > thread (like Bernd and I did, during this past hour[2]), those > comments aren't being registered in that github PR. This used to work > previously and seems to be broken right now. > > [1] > https://urldefense.com/v3/__https://github.com/openjdk/jdk/pull/4607__;!!ACWV5N9M2RV99hQ!eNjD04SNNH9INHTt0cyyed12lRNw9aZfSspNKh0ILIUQCIcvka-_vJ8XHXipGIQ$ > > [2] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-July/079994.html > > > -Jaikiran > > On 22/07/21 3:03 am, tim.bell at oracle.com wrote: >>> just recognized that all of a sudden the traffic on the OpenJDK >>> mailing lists dropped >> >> About an hour ago we corrected a problem on the Skara server >> processing external (EG: github) messages.? Message traffic should be >> flowing again. >> >> Messages for events during the outage are lost.? We will be holding a >> port-mortem to investigate why this outage was not noticed sooner, >> and why there was a data loss. >> >> Tim >> >> >> On 7/21/21 14:29, David Holmes wrote: >>> Hi Christoph, >>> >>> On 22/07/2021 1:07 am, Langer, Christoph wrote: >>>> Hi, >>>> >>>> there's still no traffic on the mailing lists... >>>> >>>> It would be much appreciated if you could share an update on the >>>> status. >>> >>> There was no update to share I'm afraid but it has now been resolved. >>> >>> It seems likely that the missing mail is gone forever unfortunately. >>> >>> Cheers, >>> David >>> >>>> Thanks >>>> Christoph >>>> >>>> From: Langer, Christoph >>>> Sent: Dienstag, 20. Juli 2021 08:23 >>>> To: ops at openjdk.java.net; skara-dev at openjdk.java.net; >>>> jdk-dev at openjdk.java.net >>>> Subject: GitHub/Mailing list communication defunct? >>>> >>>> Hi, >>>> >>>> I just recognized that all of a sudden the traffic on the OpenJDK >>>> mailing lists dropped... First I thought it's an issue with my >>>> account but looking at the pipermail archives, I don't find any >>>> mails after e.g. some time yesterday. And I guess there's still the >>>> usual activity on GitHub... >>>> >>>> Are you aware of an issue? >>>> >>>> Thanks & Best regards >>>> Christoph >>>> >> > From peter.firmstone at zeus.net.au Tue Jul 27 23:12:32 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Wed, 28 Jul 2021 09:12:32 +1000 Subject: How to remove the SecurityManager In-Reply-To: <2139489865.1124.1627415558046.JavaMail.zimbra@u-pem.fr> References: <960217215.42388.1627402274612.JavaMail.zimbra@u-pem.fr> <92effb58-0218-82ca-afaf-6c9014572fad@oracle.com> <2139489865.1124.1627415558046.JavaMail.zimbra@u-pem.fr> Message-ID: Thanks Remi, Sand-boxing is a bad idea, we are in agreement, it's not something we do, personally I'm taking an interest in safer languages, eg Haskell on secure platforms, eg OpenBSD on Sparc64 *. Perhaps JEP 411 is simply a reflection on the evolution of languages.? Java was safer than C and C++ so replaced these, something safer again will replace Java. I think people are getting our primary use case, authorization, confused with sandboxing (not on our use case list).? OpenJDK developers provided a Sandbox example, I just wanted to communicate that I didn't think it was a practical defense against exploits, nor applicable to our use case: https://inside.java/2021/04/23/security-and-sandboxing-post-securitymanager/ Our process for establishing whether third party libraries are trusted before we use them: 1. Build dependency check using Owasp https://owasp.org/www-project-dependency-check/? Reject any dependencies that fail, see https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/pom.xml line 87 for an example of a disabled module due to a vulnerability in a dependency, the module will only be re-enabled if the vulnerability is fixed. 2. Static analysis using SpotBugs, then review identified bugs, review source code if available.? Reject if security bugs are present, or fix / patch. 3. Profiling of permission access checks using: https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java 4. Reviewing generated policy files, using grep, this example was generated from over 2000 tests: https://github.com/pfirmstone/JGDMS/blob/trunk/qa/harness/policy/defaultsecuresharedvm.policy.new 5. Remove any permission from the policy file you don't want to grant to third party code, if safe to do so, eg usage statistics reporting. One of my use cases for SM is for auditing to establish trust, and then using SM with POLP policy files generated following the audit, to turn off JVM features we're not using.?? Our policy provider is performant and high scaling even with policy files containing 1000's of lines: https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/org/apache/river/api/security/ConcurrentPolicyFile.java Our use of SM for access decisions occurs during and after authentication, but also defines access roles for trusted parties, it's not possible to replace SM authorization layer functionality (not to be confused with sandboxes).?? Our use case is distributed systems, with trusted services and trusted clients, which have POJO proxy's, different service proxies are given different ProtectionDomain identity and these identities are used for authorization decisions. In a simple Client - Server application, you only have one user, from the client and the thread runs with this permission, but our systems might be performing a transaction, with 5 different services, and the transaction service is the client of these 5 services, which are represented by their proxy ProtectionDomain's.?? If one of the authenticated services is not authorized to participate in the transaction (eg a third party that's not on the contract, or maybe the contract expired), then it's not authorized and the transaction will fail.? This all occurs over secure authenticated connections, where both servers and clients are authenticated, who's the server and who's the client, well that gets a little blurred sometimes. https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/transaction/Transaction.java Back in the Jini days, Sun Microsystems, allowed different service proxy's to be loaded by the same ClassLoader, if they had the same CodeSource, they had the same identity if they had the same parent ClassLoader, we don't do that, ClassLoader's are assigned to a service proxy, based on it's authenticated identity. https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-pref-class-loader/src/main/java/net/jini/loader/pref/PreferredProxyCodebaseProvider.java This system, at its foundations is based on Jini Extensible Remote Invocation (JERI), we've replaced the serialization layer, to use what we term atomic serialization and apply constraints during connection establishment over secure connections. https://github.com/pfirmstone/JGDMS/tree/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/constraint We limit access based on both the service and user identity.? We generate our policy files by profiling (the tool creates a policy file with correct syntax, ready for immediate use), we recently added replacement of local file paths with properties for policy property expansion with cross platform trans-portability.? While its possible to use a dynamic proxy without downloading code, via an atomic serialization connection, it's not generally advised to do so with unauthenticated users, decisions around dynamic discovery, whether class loading or downloads are allowed, it's all based on policy decisions. The problem with our software is its designed to operate on un-trusted networks, and SM infrastructure is involved in authorization decisions during the authentication process, as well as providing user credentials for secure connections. We have no future Java migration path after JEP 411, the decision's been made, time to move on... On the bright side, according the JEP 411, we did achieve what OpenJDK dev's thought to be almost impossible. :)?? I'm pretty sure using the process I've documented above, you will identify 99% of accidental vulnerabilities in local code, and that was good enough for me lol. > The threat of accidental vulnerabilities in local code is almost > impossible to address with the Security Manager. * OpenBSD on Sparc (very well supported, Oracle should sell these lol, the only drawback is no zfs) is a good idea, no Spectre or Meltdown vulnerabilities. buffy$ uname -a OpenBSD buffy.lan 6.7 GENERIC.MP#310 sparc64 Although this one's a couple of versions behind, time for an upgrade. Regards, Peter. On 28/07/2021 5:52 am, forax at univ-mlv.fr wrote: > ----- Original Message ----- >> From: "Alan Bateman" >> To: "Remi Forax" , "Peter Firmstone" >> Sent: Tuesday, July 27, 2021 6:33:25 PM >> Subject: Re: How to remove the SecurityManager >> On 27/07/2021 17:11, Remi Forax wrote: >>> Peter, this is how you remove the security manager using the jdk 17 (the >>> SystemMirror class is specific to a JDK version). >>> >>> Any in-process security measures fail if the API let you to peek and poke the >>> memory like Unsafe does. >> I hope you aren't really suggesting anyone does this :-) > nope, it's a small example to explain why in-process sandboxing is a bad idea. > > >> It's dependent >> on the field layout so can break and crash the VM if it doesn't match. >> Also it assumes that someone gets theUnsafe before a SM is set. > yes, it's just an example, you have infinite variations using JNI/JNA/JNR or panama and changing some field value. > >> -Alan > R?mi From peter.firmstone at zeus.net.au Tue Jul 27 23:52:19 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Wed, 28 Jul 2021 09:52:19 +1000 Subject: How to remove the SecurityManager In-Reply-To: References: <960217215.42388.1627402274612.JavaMail.zimbra@u-pem.fr> <92effb58-0218-82ca-afaf-6c9014572fad@oracle.com> <2139489865.1124.1627415558046.JavaMail.zimbra@u-pem.fr> Message-ID: On 28/07/2021 9:12 am, Peter Firmstone wrote: > While its possible to use a dynamic proxy without downloading code, > via an atomic serialization connection, it's not generally advised to > do so with unauthenticated users, decisions around dynamic discovery, > whether class loading or downloads are allowed, it's all based on > policy decisions. Minor clarification / correction, it's not possible on our system to allow an unauthenticated user over a secure connection, our code disallows TLS connections with anon clients. We do provide TCP/IP connections, that are unsecured, however this is generally to allow testing of services during development and shouldn't be used in production.?? No changes to a service need to be made other than configuration settings to enable secure connections. Regards, Peter. From adinn at redhat.com Wed Jul 28 09:22:40 2021 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 28 Jul 2021 10:22:40 +0100 Subject: Result: New JDK Reviewer: Nick Gasson Message-ID: <18e609b2-fa49-10d8-b4e3-f6c5bfafc599@redhat.com> Voting for Nick Gasson [1] is now closed. Yes: 14 Veto: 0 Abstain: 0 According to the Bylaws definition of Three-Vote Consensus, this is sufficient to approve the nomination. Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill [1] https://mail.openjdk.java.net/pipermail/jdk-dev/2021-July/005753.html From forax at univ-mlv.fr Wed Jul 28 09:41:08 2021 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 28 Jul 2021 11:41:08 +0200 (CEST) Subject: How to remove the SecurityManager In-Reply-To: References: <960217215.42388.1627402274612.JavaMail.zimbra@u-pem.fr> <92effb58-0218-82ca-afaf-6c9014572fad@oracle.com> <2139489865.1124.1627415558046.JavaMail.zimbra@u-pem.fr> Message-ID: <199834340.71514.1627465268614.JavaMail.zimbra@u-pem.fr> > From: "Peter Firmstone" > To: "Remi Forax" , "Alan Bateman" > Cc: "jdk-dev" , "security-dev" > > Sent: Wednesday, July 28, 2021 1:12:32 AM > Subject: Re: How to remove the SecurityManager > Thanks Remi, > Sand-boxing is a bad idea, we are in agreement, it's not something we do, > personally I'm taking an interest in safer languages, eg Haskell on secure > platforms, eg OpenBSD on Sparc64 *. > Perhaps JEP 411 is simply a reflection on the evolution of languages. Java was > safer than C and C++ so replaced these, something safer again will replace > Java. All mainstream languages have a way to access to raw pointers to be able to call C functions, here is the one in Haskell https://hackage.haskell.org/package/base-4.5.0.0/docs/Foreign-Storable.html > I think people are getting our primary use case, authorization, confused with > sandboxing (not on our use case list). OpenJDK developers provided a Sandbox > example, I just wanted to communicate that I didn't think it was a practical > defense against exploits, nor applicable to our use case: > [ https://inside.java/2021/04/23/security-and-sandboxing-post-securitymanager/ | > https://inside.java/2021/04/23/security-and-sandboxing-post-securitymanager/ ] > Our process for establishing whether third party libraries are trusted before we > use them: > 1. Build dependency check using Owasp [ > https://owasp.org/www-project-dependency-check/ | > https://owasp.org/www-project-dependency-check/ ] Reject any dependencies that > fail, see [ https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/pom.xml | > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/pom.xml ] line 87 for an > example of a disabled module due to a vulnerability in a dependency, the module > will only be re-enabled if the vulnerability is fixed. > 2. Static analysis using SpotBugs, then review identified bugs, review source > code if available. Reject if security bugs are present, or fix / patch. > 3. Profiling of permission access checks using: [ > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java > | > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java > ] > 4. Reviewing generated policy files, using grep, this example was generated from > over 2000 tests: [ > https://github.com/pfirmstone/JGDMS/blob/trunk/qa/harness/policy/defaultsecuresharedvm.policy.new > | > https://github.com/pfirmstone/JGDMS/blob/trunk/qa/harness/policy/defaultsecuresharedvm.policy.new > ] > 5. Remove any permission from the policy file you don't want to grant to third > party code, if safe to do so, eg usage statistics reporting. > One of my use cases for SM is for auditing to establish trust, and then using SM > with POLP policy files generated following the audit, to turn off JVM features > we're not using. Our policy provider is performant and high scaling even with > policy files containing 1000's of lines: [ > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/org/apache/river/api/security/ConcurrentPolicyFile.java > | > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/org/apache/river/api/security/ConcurrentPolicyFile.java > ] > Our use of SM for access decisions occurs during and after authentication, but > also defines access roles for trusted parties, it's not possible to replace SM > authorization layer functionality (not to be confused with sandboxes). Our use > case is distributed systems, with trusted services and trusted clients, which > have POJO proxy's, different service proxies are given different > ProtectionDomain identity and these identities are used for authorization > decisions. > In a simple Client - Server application, you only have one user, from the client > and the thread runs with this permission, but our systems might be performing a > transaction, with 5 different services, and the transaction service is the > client of these 5 services, which are represented by their proxy > ProtectionDomain's. If one of the authenticated services is not authorized to > participate in the transaction (eg a third party that's not on the contract, or > maybe the contract expired), then it's not authorized and the transaction will > fail. This all occurs over secure authenticated connections, where both servers > and clients are authenticated, who's the server and who's the client, well that > gets a little blurred sometimes. > [ > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/transaction/Transaction.java > | > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/transaction/Transaction.java > ] > Back in the Jini days, Sun Microsystems, allowed different service proxy's to be > loaded by the same ClassLoader, if they had the same CodeSource, they had the > same identity if they had the same parent ClassLoader, we don't do that, > ClassLoader's are assigned to a service proxy, based on it's authenticated > identity. > [ > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-pref-class-loader/src/main/java/net/jini/loader/pref/PreferredProxyCodebaseProvider.java > | > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-pref-class-loader/src/main/java/net/jini/loader/pref/PreferredProxyCodebaseProvider.java > ] > This system, at its foundations is based on Jini Extensible Remote Invocation > (JERI), we've replaced the serialization layer, to use what we term atomic > serialization and apply constraints during connection establishment over secure > connections. [ > https://github.com/pfirmstone/JGDMS/tree/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/constraint > | > https://github.com/pfirmstone/JGDMS/tree/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/constraint > ] > We limit access based on both the service and user identity. We generate our > policy files by profiling (the tool creates a policy file with correct syntax, > ready for immediate use), we recently added replacement of local file paths > with properties for policy property expansion with cross platform > trans-portability. While its possible to use a dynamic proxy without > downloading code, via an atomic serialization connection, it's not generally > advised to do so with unauthenticated users, decisions around dynamic > discovery, whether class loading or downloads are allowed, it's all based on > policy decisions. > The problem with our software is its designed to operate on un-trusted networks, > and SM infrastructure is involved in authorization decisions during the > authentication process, as well as providing user credentials for secure > connections. > We have no future Java migration path after JEP 411, the decision's been made, > time to move on... > On the bright side, according the JEP 411, we did achieve what OpenJDK dev's > thought to be almost impossible. :) I'm pretty sure using the process I've > documented above, you will identify 99% of accidental vulnerabilities in local > code, and that was good enough for me lol. >> The threat of accidental vulnerabilities in local code is almost impossible to >> address with the Security Manager. In your validation process, you have a static part to check the dependencies + SpotBug and a runtime part using a combination of class loader + security manager. For the runtime part, instead of using classloaders, you can use an agent, it will also see all the requests to load a class, it can then do a static analysis of the bytecode to determine if the bytecode only contains kosher method calls and field access, the same way SpotBug does. If you really want to have a mechanism that authorize some method calls or not at runtime, you can change the bytecode to introduce a method call that checks the security policy just before the authorizable method call/field access (you also have to blacklist java.lang.reflect and java.lang.invoke but i supppose you already do this). This approach is better than using a classloader + security manager because - Java allows you to define classes not linked to a classloader since Java 8 (the old API is Unsafe.defineAnonymousClass(), the new one is Lookup.defineHiddenClass()) - you can check any calls not only the ones that the SecurityManager traps. - you can reject calls before loading the class, so earlier than with a SecurityManager, more like the bytecode verifier does. - it's more lightweight in term of memory usage because it does not rely on ClassLoaders (each ClassLoader has its own metaspace, so a lot of CL fragment the memory a lot). To read and transform the bytecode, you can ASM [1], this is one of the library used by SpotBug to read/check the bytecode. (disclaimer: i'm one of the maintainer of that library). It's still not 100% perfect because the agent runs in the same process as the code. (you can go deeper by having the authorization framework in a VM puppeteering a client VM likes jshell does using JVMTI). > * OpenBSD on Sparc (very well supported, Oracle should sell these lol, the only > drawback is no zfs) is a good idea, no Spectre or Meltdown vulnerabilities. > buffy$ uname -a > OpenBSD buffy.lan 6.7 GENERIC.MP#310 sparc64 > Although this one's a couple of versions behind, time for an upgrade. > Regards, > Peter. regards, R?mi [1] https://asm.ow2.io/ > On 28/07/2021 5:52 am, [ mailto:forax at univ-mlv.fr | forax at univ-mlv.fr ] wrote: >> ----- Original Message ----- >>> From: "Alan Bateman" [ mailto:Alan.Bateman at oracle.com | >>> ] To: "Remi Forax" [ mailto:forax at univ-mlv.fr | >>> ] , "Peter Firmstone" [ mailto:peter.firmstone at zeus.net.au >>> | ] Sent: Tuesday, July 27, 2021 6:33:25 PM >>> Subject: Re: How to remove the SecurityManager >>> On 27/07/2021 17:11, Remi Forax wrote: >>>> Peter, this is how you remove the security manager using the jdk 17 (the >>>> SystemMirror class is specific to a JDK version). >>>> Any in-process security measures fail if the API let you to peek and poke the >>>> memory like Unsafe does. >>> I hope you aren't really suggesting anyone does this :-) >> nope, it's a small example to explain why in-process sandboxing is a bad idea. >>> It's dependent >>> on the field layout so can break and crash the VM if it doesn't match. >>> Also it assumes that someone gets theUnsafe before a SM is set. >> yes, it's just an example, you have infinite variations using JNI/JNA/JNR or >> panama and changing some field value. >>> -Alan >> R?mi From peter.firmstone at zeus.net.au Wed Jul 28 12:10:30 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Wed, 28 Jul 2021 22:10:30 +1000 Subject: How to remove the SecurityManager In-Reply-To: <199834340.71514.1627465268614.JavaMail.zimbra@u-pem.fr> References: <960217215.42388.1627402274612.JavaMail.zimbra@u-pem.fr> <92effb58-0218-82ca-afaf-6c9014572fad@oracle.com> <2139489865.1124.1627415558046.JavaMail.zimbra@u-pem.fr> <199834340.71514.1627465268614.JavaMail.zimbra@u-pem.fr> Message-ID: <25541dcc-e1d2-cc6f-1770-7d3ebcea24f4@zeus.net.au> Thanks Remi, I'm a user of ASM also, for a long time, since 2007, it's a very performant library. Yes, we could replace the policy audit with another tool, but it's academic, the remaining code cannot be upgraded. For now the policy tools informs me of reflection access, I don't need to blacklist it if I read the code and it's doing something harmless, eg. it might be calling public methods, to support multiple versions of java. I looked at Agents to replace permission checks, it requires modification of private methods, it's bad practice, we've removed all code that accessed private implementation or state, we only use public API. It's not just a simple case of instrumenting public API's, many permissions defend constructors, and constructors contain private static methods to defend against finalizer attacks.?? While I could defend public methods, methods are called far more often than constructors, it would have an unacceptable impact on performance.? Years will pass before finalizers are removed and constructors are simplified so they can be instrumented. It's not viable to re-implement an authorization layer as an external library for Java. Right now SM only has a less than 3% impact on performance and doesn't affect scalability, how can I justify replacing it, for what new feature??? I don't run untrusted code, it works reliably for the authorization based access controls that I require and provides access to subject credentials for authentication of secure connections. Performance profiling of SM running with stateless TLS sockets https://imgur.com/VcSwffC https://imgur.com/VcSwffC https://imgur.com/VcSwffC I think Haskell has better type safety than Java, it handles Null with Maybe, it's good for parsing data, it appears to have made few compromises in its design, but I'm not saying that from experience. I think if I was looking for something to run untrusted code, it would be as source code that I parsed, then compiled, perhaps a subset of Haskell parsed as source code, if I used it for that, then it's audited by parsing and the compiler. I guess something similar could be done with ASM and bytecode, but it's not my goal to run untrusted code, I'll leave the sandbox for the developers cat to bury applets. Regards, Peter. On 28/07/2021 7:41 pm, forax at univ-mlv.fr wrote: > > > ------------------------------------------------------------------------ > > *From: *"Peter Firmstone" > *To: *"Remi Forax" , "Alan Bateman" > > *Cc: *"jdk-dev" , "security-dev" > > *Sent: *Wednesday, July 28, 2021 1:12:32 AM > *Subject: *Re: How to remove the SecurityManager > > Thanks Remi, > > Sand-boxing is a bad idea, we are in agreement, it's not something > we do, personally I'm taking an interest in safer languages, eg > Haskell on secure platforms, eg OpenBSD on Sparc64 *. > > Perhaps JEP 411 is simply a reflection on the evolution of > languages.? Java was safer than C and C++ so replaced these, > something safer again will replace Java. > > > All mainstream languages have a way to access to raw pointers to be > able to call C functions, > here is the one in Haskell > https://hackage.haskell.org/package/base-4.5.0.0/docs/Foreign-Storable.html > > > I think people are getting our primary use case, authorization, > confused with sandboxing (not on our use case list).? OpenJDK > developers provided a Sandbox example, I just wanted to > communicate that I didn't think it was a practical defense against > exploits, nor applicable to our use case: > > https://inside.java/2021/04/23/security-and-sandboxing-post-securitymanager/ > > > Our process for establishing whether third party libraries are > trusted before we use them: > > 1. Build dependency check using Owasp > https://owasp.org/www-project-dependency-check/ > Reject any > dependencies that fail, see > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/pom.xml > > line 87 for an example of a disabled module due to a > vulnerability in a dependency, the module will only be > re-enabled if the vulnerability is fixed. > 2. Static analysis using SpotBugs, then review identified bugs, > review source code if available.? Reject if security bugs are > present, or fix / patch. > 3. Profiling of permission access checks using: > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java > > 4. Reviewing generated policy files, using grep, this example was > generated from over 2000 tests: > https://github.com/pfirmstone/JGDMS/blob/trunk/qa/harness/policy/defaultsecuresharedvm.policy.new > > 5. Remove any permission from the policy file you don't want to > grant to third party code, if safe to do so, eg usage > statistics reporting. > > One of my use cases for SM is for auditing to establish trust, and > then using SM with POLP policy files generated following the > audit, to turn off JVM features we're not using.?? Our policy > provider is performant and high scaling even with policy files > containing 1000's of lines: > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/org/apache/river/api/security/ConcurrentPolicyFile.java > > > Our use of SM for access decisions occurs during and after > authentication, but also defines access roles for trusted parties, > it's not possible to replace SM authorization layer functionality > (not to be confused with sandboxes).?? Our use case is distributed > systems, with trusted services and trusted clients, which have > POJO proxy's, different service proxies are given different > ProtectionDomain identity and these identities are used for > authorization decisions. > > In a simple Client - Server application, you only have one user, > from the client and the thread runs with this permission, but our > systems might be performing a transaction, with 5 different > services, and the transaction service is the client of these 5 > services, which are represented by their proxy ProtectionDomain's. > If one of the authenticated services is not authorized to > participate in the transaction (eg a third party that's not on the > contract, or maybe the contract expired), then it's not authorized > and the transaction will fail.? This all occurs over secure > authenticated connections, where both servers and clients are > authenticated, who's the server and who's the client, well that > gets a little blurred sometimes. > > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/transaction/Transaction.java > > > Back in the Jini days, Sun Microsystems, allowed different service > proxy's to be loaded by the same ClassLoader, if they had the same > CodeSource, they had the same identity if they had the same parent > ClassLoader, we don't do that, ClassLoader's are assigned to a > service proxy, based on it's authenticated identity. > > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/jgdms-pref-class-loader/src/main/java/net/jini/loader/pref/PreferredProxyCodebaseProvider.java > > > This system, at its foundations is based on Jini Extensible Remote > Invocation (JERI), we've replaced the serialization layer, to use > what we term atomic serialization and apply constraints during > connection establishment over secure connections. > > https://github.com/pfirmstone/JGDMS/tree/trunk/JGDMS/jgdms-platform/src/main/java/net/jini/core/constraint > > > We limit access based on both the service and user identity.? We > generate our policy files by profiling (the tool creates a policy > file with correct syntax, ready for immediate use), we recently > added replacement of local file paths with properties for policy > property expansion with cross platform trans-portability.? While > its possible to use a dynamic proxy without downloading code, via > an atomic serialization connection, it's not generally advised to > do so with unauthenticated users, decisions around dynamic > discovery, whether class loading or downloads are allowed, it's > all based on policy decisions. > > The problem with our software is its designed to operate on > un-trusted networks, and SM infrastructure is involved in > authorization decisions during the authentication process, as well > as providing user credentials for secure connections. > > We have no future Java migration path after JEP 411, the > decision's been made, time to move on... > > On the bright side, according the JEP 411, we did achieve what > OpenJDK dev's thought to be almost impossible. :) I'm pretty sure > using the process I've documented above, you will identify 99% of > accidental vulnerabilities in local code, and that was good enough > for me lol. > > The threat of accidental vulnerabilities in local code is > almost impossible to address with the Security Manager. > > > In your validation process, you have a static part to check the > dependencies + SpotBug and a runtime part using a combination of class > loader + security manager. > For the runtime part, instead of using classloaders, you can use an > agent, it will also see all the requests to load a class, it can then > do a static analysis of the bytecode to determine if the bytecode only > contains kosher method calls and field access, the same way SpotBug does. > If you really want to have a mechanism that authorize some method > calls or not at runtime, you can change the bytecode to introduce a > method call that checks the security policy just before the > authorizable method call/field access > (you also have to blacklist java.lang.reflect and java.lang.invoke but > i supppose you already do this). > > This approach is better than using a classloader + security manager > because > - Java allows you to define classes not linked to a classloader since > Java 8 (the old API is Unsafe.defineAnonymousClass(), the new one is > Lookup.defineHiddenClass()) > - you can check any calls not only the ones that the SecurityManager > traps. > - you can reject calls before loading the class, so earlier than with > a SecurityManager, more like the bytecode verifier does. > - it's more lightweight in term of memory usage because it does not > rely on ClassLoaders (each ClassLoader has its own metaspace, so a lot > of CL fragment the memory a lot). > > To read and transform the bytecode, you can ASM [1], this is one of > the library used by SpotBug to read/check the bytecode. > (disclaimer: i'm one of the maintainer of that library). > > It's still not 100% perfect because the agent runs in the same process > as the code. > (you can go deeper by having the authorization framework in a VM > puppeteering a client VM likes jshell does using JVMTI). > > > * OpenBSD on Sparc (very well supported, Oracle should sell these > lol, the only drawback is no zfs) is a good idea, no Spectre or > Meltdown vulnerabilities. > > buffy$ uname -a > OpenBSD buffy.lan 6.7 GENERIC.MP#310 sparc64 > > Although this one's a couple of versions behind, time for an upgrade. > > Regards, > > Peter. > > > regards, > R?mi > > [1] https://asm.ow2.io/ > > > On 28/07/2021 5:52 am, forax at univ-mlv.fr > wrote: > > ----- Original Message ----- > > From: "Alan Bateman" > To: "Remi Forax" , "Peter Firmstone" > Sent: Tuesday, July 27, 2021 6:33:25 PM > Subject: Re: How to remove the SecurityManager > > On 27/07/2021 17:11, Remi Forax wrote: > > Peter, this is how you remove the security manager using the jdk 17 (the > SystemMirror class is specific to a JDK version). > > Any in-process security measures fail if the API let you to peek and poke the > memory like Unsafe does. > > I hope you aren't really suggesting anyone does this :-) > > nope, it's a small example to explain why in-process sandboxing is a bad idea. > > > It's dependent > on the field layout so can break and crash the VM if it doesn't match. > Also it assumes that someone gets theUnsafe before a SM is set. > > yes, it's just an example, you have infinite variations using JNI/JNA/JNR or panama and changing some field value. > > -Alan > > R?mi > > From peter.firmstone at zeus.net.au Thu Jul 29 01:20:49 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Thu, 29 Jul 2021 11:20:49 +1000 Subject: How to remove the SecurityManager In-Reply-To: References: <960217215.42388.1627402274612.JavaMail.zimbra@u-pem.fr> <92effb58-0218-82ca-afaf-6c9014572fad@oracle.com> <2139489865.1124.1627415558046.JavaMail.zimbra@u-pem.fr> Message-ID: <7164cfa7-d76a-5de0-dcee-1eda919bcfc4@zeus.net.au> The intent of the following process is to perform a targeted audit, which allows inspection of small parts of the code identified by these steps. On 28/07/2021 9:12 am, Peter Firmstone wrote: > > Our process for establishing whether third party libraries are trusted > before we use them: > > 1. Build dependency check using Owasp > https://owasp.org/www-project-dependency-check/ Reject any > dependencies that fail, see > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/pom.xml line > 87 for an example of a disabled module due to a vulnerability in a > dependency, the module will only be re-enabled if the > vulnerability is fixed. > 2. Static analysis using SpotBugs, then review identified bugs, > review source code if available.? Reject if security bugs are > present, or fix / patch. > 3. Profiling of permission access checks using: > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java > 4. Reviewing generated policy files, using grep, this example was > generated from over 2000 tests: > https://github.com/pfirmstone/JGDMS/blob/trunk/qa/harness/policy/defaultsecuresharedvm.policy.new > 5. Remove any permission from the policy file you don't want to grant > to third party code, if safe to do so, eg usage statistics reporting. > In the construction industry, a similar approach is used by structural engineers and weld inspectors, when inspecting welds for defects, using Ultrasonic, X-Ray or visual inspection, weld defects in structures have the potential to cause catastrophic failure and multiple fatalities, likely worse consequences than a bug in Java, so engineers identify critical areas? for inspectors to target with 100% coverage, perhaps by UT or X-Ray, to inspect the weld internally, then the structural engineer will nominate to inspect 10% of other areas with UT, with 100% visual inspection, for example, if defects are found, then they will increase UT inspection coverage, welds need to be gouged out and re-welded, until the inspector is satisfied with quality. A targeted code audit process will also identify code quality, if there are many bugs, don't use it, even if these aren't security bugs. This can hardly be compared with the approach used for running Applets in a sandbox that may have malicious intent, in that case no auditing has been performed at all. This use case of SecurityManager recognizes shortcomings of Java platform security.?? Sandboxing was a marketing term used for Applets, I don't know the origin of the term sandbox when used for computer security, but whenever there is a sandbox, there is a risk of escape, and simplicity is thy friend, it should be left to cyber security professionals. If you removed applets, then there is no use case for a Sandbox, so why remove SecurityManager? Come on honestly, JEP 411 is confirmation biased, is overly focused on sandboxing and therefore not factual or relevant, I've provided sufficient evidence contrary to it's claims.?? It needs to take the ability to migrate code into account as well as use cases other than sandboxing. We use SM to prevent loading of untrusted (unaudited) code and untrusted (unauthenticated) data, but we don't use it as a sandbox to attempt to encapsulate malicious code and malicious users, we use it for authorization decisions, for external users and services, this could also be applied to Web Services, not just Jini services, these authorization decisions prevent loading untrusted code and parsing untrusted data. grant codebase "httpmd://${HOST}:9080/${mercury-dl.jar};sha-384=041531e5e3de288c121e865af8a46f7af86172ee0127dc4aff4f551c73a0ad604f51cb1c53076140fd415f957c14e8dd", ??? principal javax.security.auth.x500.X500Principal "CN=Outrigger" { ??? permission org.apache.river.api.io.DeSerializationPermission "MARSHALL"; ??? permission net.jini.security.AuthenticationPermission "javax.security.auth.x500.X500Principal \"CN=Outrigger\" peer javax.security.auth.x500.X500Principal \"CN=Phoenix\"", "connect"; }; Rather than throwing developers who use SM under the bus, we could be given a migration path: 1. Review and reduce the number of permissions focused *only* on authorization use cases. ? Eg: Give Properties useful for authorization their own guarded area in the Property map??? I mean, why are we guarding java.util.PropertyPermission "java.specification.version", "read" and many others like it? Fix SocketPermission, add netmask wild cards, use RFC3986 normalization, stop using DNS.? Ever heard of DNS spoofing? 2. What about parsing of data?? Such as XML and Java Serialization, among others, this should have a permission check, that when granted to users, ensures the data source has been authenticated.?? This is a server application, not client.?? Permission to parse data should only be granted to user principal's.? No user, then the data is untrusted and shouldn't be parsed. 3. Create a guard security provider interface to replace permission checks, to allow developers to focus on their authorization needs, that would allow us to completely ignore permission checks that are irrelevant and replace bad implementations like SocketPermission. 4. Consider simplification of "Voodoo", maybe instead of trying to check every Thread stack (inheriting call stacks), if there is no doPrivileged call on the current Thread's stack (reduces shared state, it's thread confined), then report it and throw a SecurityException, don't try to inherit thread context, because it doesn't work for executor tasks.?? Then it will be fixed downstream instead of allowed to create viral permission checks that violate the principle of least privilege.?? This is not a security vulnerability risk, we are only using it for authorization, not sandboxing and it will make policy files much shorter, improving readability. 5. We still use AccessController and AccessControlContext to establish TLS connections, why break it? 6. Get a tool to generate policy files (mine's only 653 lines of code). https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java JEP 411 nukes backward compatibility so it cannot be fixed at all. Regards, Peter. From peter.firmstone at zeus.net.au Thu Jul 29 06:38:23 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Thu, 29 Jul 2021 16:38:23 +1000 Subject: How to remove the SecurityManager In-Reply-To: <7164cfa7-d76a-5de0-dcee-1eda919bcfc4@zeus.net.au> References: <960217215.42388.1627402274612.JavaMail.zimbra@u-pem.fr> <92effb58-0218-82ca-afaf-6c9014572fad@oracle.com> <2139489865.1124.1627415558046.JavaMail.zimbra@u-pem.fr> <7164cfa7-d76a-5de0-dcee-1eda919bcfc4@zeus.net.au> Message-ID: Appended inline below. On 29/07/2021 11:20 am, Peter Firmstone wrote: > > The intent of the following process is to perform a targeted audit, > which allows inspection of small parts of the code identified by these > steps. > > On 28/07/2021 9:12 am, Peter Firmstone wrote: >> >> Our process for establishing whether third party libraries are >> trusted before we use them: >> >> 1. Build dependency check using Owasp >> https://owasp.org/www-project-dependency-check/ Reject any >> dependencies that fail, see >> https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/pom.xml line >> 87 for an example of a disabled module due to a vulnerability in >> a dependency, the module will only be re-enabled if the >> vulnerability is fixed. >> 2. Static analysis using SpotBugs, then review identified bugs, >> review source code if available.? Reject if security bugs are >> present, or fix / patch. >> 3. Profiling of permission access checks using: >> https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java >> 4. Reviewing generated policy files, using grep, this example was >> generated from over 2000 tests: >> https://github.com/pfirmstone/JGDMS/blob/trunk/qa/harness/policy/defaultsecuresharedvm.policy.new >> 5. Remove any permission from the policy file you don't want to >> grant to third party code, if safe to do so, eg usage statistics >> reporting. >> > In the construction industry, a similar approach is used by structural > engineers and weld inspectors, when inspecting welds for defects, > using Ultrasonic, X-Ray or visual inspection, weld defects in > structures have the potential to cause catastrophic failure and > multiple fatalities, likely worse consequences than a bug in Java, so > engineers identify critical areas? for inspectors to target with 100% > coverage, perhaps by UT or X-Ray, to inspect the weld internally, then > the structural engineer will nominate to inspect 10% of other areas > with UT, with 100% visual inspection, for example, if defects are > found, then they will increase UT inspection coverage, welds need to > be gouged out and re-welded, until the inspector is satisfied with > quality. > > A targeted code audit process will also identify code quality, if > there are many bugs, don't use it, even if these aren't security bugs. > > This can hardly be compared with the approach used for running Applets > in a sandbox that may have malicious intent, in that case no auditing > has been performed at all. > > This use case of SecurityManager recognizes shortcomings of Java > platform security.?? Sandboxing was a marketing term used for Applets, > I don't know the origin of the term sandbox when used for computer > security, but whenever there is a sandbox, there is a risk of escape, > and simplicity is thy friend, it should be left to cyber security > professionals. > > If you removed applets, then there is no use case for a Sandbox, so > why remove SecurityManager? > > Come on honestly, JEP 411 is confirmation biased, is overly focused on > sandboxing and therefore not factual or relevant, I've provided > sufficient evidence contrary to it's claims.?? It needs to take the > ability to migrate code into account as well as use cases other than > sandboxing. > > We use SM to prevent loading of untrusted (unaudited) code and > untrusted (unauthenticated) data, but we don't use it as a sandbox to > attempt to encapsulate malicious code and malicious users, we use it > for authorization decisions, for external users and services, this > could also be applied to Web Services, not just Jini services, these > authorization decisions prevent loading untrusted code and parsing > untrusted data. > > grant codebase > "httpmd://${HOST}:9080/${mercury-dl.jar};sha-384=041531e5e3de288c121e865af8a46f7af86172ee0127dc4aff4f551c73a0ad604f51cb1c53076140fd415f957c14e8dd", > ??? principal javax.security.auth.x500.X500Principal "CN=Outrigger" > { > ??? permission org.apache.river.api.io.DeSerializationPermission > "MARSHALL"; > ??? permission net.jini.security.AuthenticationPermission > "javax.security.auth.x500.X500Principal \"CN=Outrigger\" peer > javax.security.auth.x500.X500Principal \"CN=Phoenix\"", "connect"; > }; > > Rather than throwing developers who use SM under the bus, we could be > given a migration path: > > 1. Review and reduce the number of permissions focused *only* on > authorization use cases. ? Eg: Give Properties useful for > authorization their own guarded area in the Property map??? I > mean, why are we guarding java.util.PropertyPermission > "java.specification.version", "read" and many others like it??? > Fix SocketPermission, add netmask wild cards, use RFC3986 > normalization, stop using DNS.? Ever heard of DNS spoofing? > 2. What about parsing of data?? Such as XML and Java Serialization, > among others, this should have a permission check, that when > granted to users, ensures the data source has been > authenticated.?? This is a server application, not client.?? > Permission to parse data should only be granted to user > principal's.? No user, then the data is untrusted and shouldn't be > parsed. > 3. Create a guard security provider interface to replace permission > checks, to allow developers to focus on their authorization needs, > that would allow us to completely ignore permission checks that > are irrelevant and replace bad implementations like SocketPermission. > 4. Consider simplification of "Voodoo", maybe instead of trying to > check every Thread stack (inheriting call stacks), if there is no > doPrivileged call on the current Thread's stack (reduces shared > state, it's thread confined), then report it and throw a > SecurityException, don't try to inherit thread context, because it > doesn't work for executor tasks.?? Then it will be fixed > downstream instead of allowed to create viral permission checks > that violate the principle of least privilege.?? This is not a > security vulnerability risk, we are only using it for > authorization, not sandboxing and it will make policy files much > shorter, improving readability. > 5. We still use AccessController and AccessControlContext to > establish TLS connections, why break it? > 6. Get a tool to generate policy files (mine's only 653 lines of > code). > https://github.com/pfirmstone/JGDMS/blob/trunk/JGDMS/tools/security-policy-debug/src/main/java/org/apache/river/tool/SecurityPolicyWriter.java > 7. https://www.oracle.com/java/technologies/javase/seccodeguide.html#8-5 Class::getProtectionDomain shouldn't return a null ProtectionDomain for Java platform library code (the java library is over-privileged), instead use modules: jrt://java.base, etc, so we can grant permissions to authenticated users instead of code, and in this case to ensure that there is an authenticated user on the stack.? The user is the data source during de-serialization. I added Security::doAs methods that appended a Subject ProtectionDomain, rather than injecting Principal's into every ProtectionDomain on the stack, this does two things, one allows a Subject to be more privileged than some code, that it interacts with and also to reduce the intersection of permissions, in the presence of privileged domains (like the Java platform libraries above), to avoid the security issues described. This method is papering over the underlying problem however, which is excessive privileges granted to code. https://github.com/pfirmstone/JGDMS/blob/abb310f4ece90e31f2444e2368efe3864d3f9c09/JGDMS/jgdms-platform/src/main/java/net/jini/security/Security.java#L590 Anyone can find problems, it's the solutions to problems that matter. 99% of problems with SM have been solved outside of OpenJDK.?? I can't see maintenance being a major issue if OpenJDK stops treating it like a sandbox, and instead, starts treating it like an authorization layer and reduce its complexity and mostly automate it.?? It doesn't require as much work as you suspect. JGDMS tests are run with security enabled by default, that includes generating TLS certificates, signing them with a test CA, then adding them to trust stores and key stores.? It's been over 10 years since I've run tests without SM enabled, I'm not even sure it works without it. > JEP 411 nukes backward compatibility so it cannot be fixed at all. > > Regards, > > Peter. > From peter.firmstone at zeus.net.au Sat Jul 31 03:04:07 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sat, 31 Jul 2021 13:04:07 +1000 Subject: JEP 411, removal of finalizers, a path forward. Message-ID: The current JEP 411 plan of action, if left unchanged, will leave developers who adopted the SM architecture as an authorization layer unable to upgrade to later versions of Java, until finalizers and the finalizer attack defensive methods in constructors are removed.? JEP 411 has the potential to cause significant disruption for a small proportion of Java developers, but doesn't have to if managed appropriately. The blocker is the ability to implement guard checks using Agents on public API, due to finalizer attack defensive private static methods in constructors. Allan has advised when finalizers are removed, it will be practical to use Agents to instrument public API to implement an authorization layer, this is try, so can it be coordinated with JEP 411 et al? Furthermore, as developers must support multiple Java releases, I propose the following amendments, to ease difficulties of multiple release support (with multi release jars): * AccessController, AccessControlContext, DomainCombiner and related Subject and Executors methods, remain until Java 8 is EOL in 2030.? Also consider un-deprecation of these methods, as their removal causes shotgun surgery (used in 1000's of locations in my software alone) and they are required for preservation of Subject, used for obtaining TLS and Kerberos connection credentials on all existing versions of Java. * AccessControlContext - remove inherited thread context, replace it with an unprivileged ProtectionDomain, such that doPrivileged methods are required for authorization checks and only the current thread stack needs to be walked when checks occur, and stack walks aren't unnecessarily performed when creating new threads.?? This is compatible with Loom, update loom to allow the use of AccessControlContext to be used, to establish TLS and Kerberos connections.? Loom will be very useful for network connections, especially long latency connections over the internet, which are typically secured using TLS.?? This removes the problem of viral checks, and Executor task privilege escalation. * Modules that are mapped to the boot loader should get a unique PD that includes a useful code source rather than using a "shared" PD, this allows us to reduce the privileged footprint of the Java platform libraries, to allow privileges to be granted to users, not code, or users and code.? This is useful to limit data parsing privileges to authenticated users on servers (a practise that should be more widely encouraged). * Remove finalizers, and defensive methods in constructors where permissions check points occur as these cause problems for Agents, prior to removal of SecurityManager. * Deprecate for removal Permission implementations, then remove them in a following release. * Remove SecurityManager. This allows a forward migration path for poor sod's like myself who are currently using SM infrastructure as an authorization layer, and to establish TLS conenctions, this or at least some sort of compromise is far preferable to the thermonuclear option currently planned. What I would like OpenJDK to consider, is to allow developers like myself to continue to stay current with Java, by coordinating the removal of finalizers and defensive methods in constructors, with JEP 411, so we have a workable future migration path. Without these considerations, options are; go back to Java 8, and plan to redevelop existing software, if forced to do so, Java is unlikely to be on the list for redevelopment, simply because development costs are lower in newer languages, such as automated unit tests, https://hackage.haskell.org/package/QuickCheck, no need to worry about null pointers and less boilerplate. Don't get me wrong, I like Java and have many years experience with it, but I have to be pragmatic, it won't just be me, many other developers, when Java 8 is EOL, will work for companies stuck on that platform, simply due to the number of changes required, because they haven't kept up (eg budgets) with the current release cadence and pace of development, will be looking at redevelopment and replacement instead of migration.?? Clearly the current pace of development is a good thing for Java, but the overall strategy could be tweaked a little, to ensure migration doesn't become insurmountable.? A healthy and vibrant Java community is essential for the survival of Java, Java has already shed phone and client markets, lets not shed too many more. Thanks, Peter From Alan.Bateman at oracle.com Sat Jul 31 07:35:06 2021 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 31 Jul 2021 08:35:06 +0100 Subject: JEP 411, removal of finalizers, a path forward. In-Reply-To: References: Message-ID: <7afa85d5-27c7-a35a-d1ed-513cf5b802ec@oracle.com> On 31/07/2021 04:04, Peter Firmstone wrote: > > Allan has advised when finalizers are removed, it will be practical to > use Agents to instrument public API to implement an authorization > layer, this is try, so can it be coordinated with JEP 411 et al? > Our exchange was about instrumenting constructors that specify SM permission checks and where the classes that define these constructors have been hardened to thwart finalizer attacks. It wasn't a comment on the bigger question on how practical it is to use instrumented the entire JDK. Once you get further on then I assume a big challenge will be with APIs that separate the interface and implementation (think factory methods, APIs that define service provider interfaces ...). Here I expect you will want to instrument the implementation classes. Going deeper, you may find places where the SM check isn't on method entry but instead after defensive copying of mutable parameters or after acquiring a lock that prevents mutation while do a security sensitive operations. So non-trivial but a fun approach to explore. If you have the cycles then pick a version and try it. That will give you a sense on how much effort may be required to keep up and be confident that every interesting code path is covered. -Alan From ron.pressler at oracle.com Sat Jul 31 08:22:47 2021 From: ron.pressler at oracle.com (Ron Pressler) Date: Sat, 31 Jul 2021 08:22:47 +0000 Subject: JEP 411, removal of finalizers, a path forward. In-Reply-To: References: Message-ID: Hi Peter. - JEP 411, like every spec-changing JEP, is meant to allow those that use the removed functionality a migration path forward. The API elements that are deprecated for removal have some years before they are actually removed, so there?s nothing too urgent other than beginning to think of a migration path. I think it?s still too soon to consider concrete suggestions for change, especially non-trivial ones. - If by Java 8 EOL you mean the time when the last vendor offers extended support for it, then 2030 is, I believe, the *earliest* possible date that is guaranteed *now. It?s possible that support would be extended until 2130. Such offerings have no bearing on the development of current JDK versions. - The number of significant code changes required since JDK 8 to keep up with current JDK releases is, for the vast majority of Java users, low (what?s affected most users is reliance on non-spec-compliant libraries, and the need to import the external artefacts for EE). The most impactful change has probably been the removal of some client deployment technologies from the Oracle JDK, but as far as OpenJDK is concerned, the affected areas have been CORBA, pack200, Nashorn, and now the process to remove SM is starting. The number of people using any one of these is low, and the number of those who need to work hard for alternatives is very low. I think that advance warning, and then support offerings by multiple vendors for those who have not managed to migrate in the advance warning period is reasonable; always offering ways to support removed technologies together with new features in current releases with the same code base is not. When compared with other ecosystems, Java?s strategy is exceptionally tolerant of those that prefer slow change. - Property-based testing is wonderful, I wish more people would use it, and I hope to see it used in the JDK as well. Java has several PBT tools, but I believe the most popular one these days is https://jqwik.net/. As long as you?re still with Java, give it a try. ? Ron > On 31 Jul 2021, at 04:04, Peter Firmstone wrote: > > The current JEP 411 plan of action, if left unchanged, will leave developers who adopted the SM architecture as an authorization layer unable to upgrade to later versions of Java, until finalizers and the finalizer attack defensive methods in constructors are removed. JEP 411 has the potential to cause significant disruption for a small proportion of Java developers, but doesn't have to if managed appropriately. > > The blocker is the ability to implement guard checks using Agents on public API, due to finalizer attack defensive private static methods in constructors. > > Allan has advised when finalizers are removed, it will be practical to use Agents to instrument public API to implement an authorization layer, this is try, so can it be coordinated with JEP 411 et al? > > Furthermore, as developers must support multiple Java releases, I propose the following amendments, to ease difficulties of multiple release support (with multi release jars): > > * AccessController, AccessControlContext, DomainCombiner and related > Subject and Executors methods, remain until Java 8 is EOL in 2030. > Also consider un-deprecation of these methods, as their removal > causes shotgun surgery (used in 1000's of locations in my software > alone) and they are required for preservation of Subject, used for > obtaining TLS and Kerberos connection credentials on all existing > versions of Java. > * AccessControlContext - remove inherited thread context, replace it > with an unprivileged ProtectionDomain, such that doPrivileged > methods are required for authorization checks and only the current > thread stack needs to be walked when checks occur, and stack walks > aren't unnecessarily performed when creating new threads. This is > compatible with Loom, update loom to allow the use of > AccessControlContext to be used, to establish TLS and Kerberos > connections. Loom will be very useful for network connections, > especially long latency connections over the internet, which are > typically secured using TLS. This removes the problem of viral > checks, and Executor task privilege escalation. > * Modules that are mapped to the boot loader should get a unique PD > that includes a useful code source rather than using a "shared" PD, > this allows us to reduce the privileged footprint of the Java > platform libraries, to allow privileges to be granted to users, not > code, or users and code. This is useful to limit data parsing > privileges to authenticated users on servers (a practise that should > be more widely encouraged). > * Remove finalizers, and defensive methods in constructors where > permissions check points occur as these cause problems for Agents, > prior to removal of SecurityManager. > * Deprecate for removal Permission implementations, then remove them > in a following release. > * Remove SecurityManager. > > This allows a forward migration path for poor sod's like myself who are currently using SM infrastructure as an authorization layer, and to establish TLS conenctions, this or at least some sort of compromise is far preferable to the thermonuclear option currently planned. > > What I would like OpenJDK to consider, is to allow developers like myself to continue to stay current with Java, by coordinating the removal of finalizers and defensive methods in constructors, with JEP 411, so we have a workable future migration path. Without these considerations, options are; go back to Java 8, and plan to redevelop existing software, if forced to do so, Java is unlikely to be on the list for redevelopment, simply because development costs are lower in newer languages, such as automated unit tests, https://hackage.haskell.org/package/QuickCheck, no need to worry about null pointers and less boilerplate. > > Don't get me wrong, I like Java and have many years experience with it, but I have to be pragmatic, it won't just be me, many other developers, when Java 8 is EOL, will work for companies stuck on that platform, simply due to the number of changes required, because they haven't kept up (eg budgets) with the current release cadence and pace of development, will be looking at redevelopment and replacement instead of migration. Clearly the current pace of development is a good thing for Java, but the overall strategy could be tweaked a little, to ensure migration doesn't become insurmountable. A healthy and vibrant Java community is essential for the survival of Java, Java has already shed phone and client markets, lets not shed too many more. > > Thanks, > > Peter > From peter.firmstone at zeus.net.au Sat Jul 31 09:05:57 2021 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sat, 31 Jul 2021 19:05:57 +1000 Subject: JEP 411, removal of finalizers, a path forward. In-Reply-To: <7afa85d5-27c7-a35a-d1ed-513cf5b802ec@oracle.com> References: <7afa85d5-27c7-a35a-d1ed-513cf5b802ec@oracle.com> Message-ID: <0366dd24-fc82-4274-d862-0913e83b307c@zeus.net.au> I'm potentially watching many years of development efforts burn, due to JEP 411 and trying to find a way to save it, I don't refer to it as fun.? Frustrating, infuriating, irritating, anger, that would more accurately describe the emotions current circumstances create. I'll be focusing only on the following in Java's public API's: 1. Network. 2. File system access. 3. ClassLoading. 4. Properties. The most important task is to prevent class loading from unauthenticated sources. This is intended to sure up perimeter defenses and constrain trusted third parties in our software, these would be wide open if we just switched off SM. I'm following all guidance provided by OpenJDK in this instance. I will wait for finalizers to be removed, before instrumenting any constructors that have finalizer attack defenses.? Hopefully OpenJDK will chose to remove finalizers soon, prior to removal of SM. Regards, Peter. On 31/07/2021 5:35 pm, Alan Bateman wrote: > On 31/07/2021 04:04, Peter Firmstone wrote: >> >> Allan has advised when finalizers are removed, it will be practical >> to use Agents to instrument public API to implement an authorization >> layer, this is try, so can it be coordinated with JEP 411 et al? >> > Our exchange was about instrumenting constructors that specify SM > permission checks and where the classes that define these constructors > have been hardened to thwart finalizer attacks. It wasn't a comment on > the bigger question on how practical it is to use instrumented the > entire JDK. Once you get further on then I assume a big challenge will > be with APIs that separate the interface and implementation (think > factory methods, APIs that define service provider interfaces ...). > Here I expect you will want to instrument the implementation classes. > Going deeper, you may find places where the SM check isn't on method > entry but instead after defensive copying of mutable parameters or > after acquiring a lock that prevents mutation while do a security > sensitive operations. So non-trivial but a fun approach to explore. If > you have the cycles then pick a version and try it. That will give you > a sense on how much effort may be required to keep up and be confident > that every interesting code path is covered. > > -Alan > >