RFR: 8262744: Formatter '%g' conversion uses wrong format for BigDecimal rounding up to limits [v2]
rriggs at openjdk.java.net
Fri Apr 16 14:30:38 UTC 2021
On Wed, 7 Apr 2021 02:48:00 GMT, Ian Graves <igraves at openjdk.org> wrote:
>> This fixes a bug where the formatting code for `%g` flags incorrectly tries to round `BigDecimal` after determining whether it should be a decimal numeric format or a scientific numeric format. The solution rounds before determining the correct format.
> Ian Graves has updated the pull request incrementally with one additional commit since the last revision:
> Adding BigDecimal equivalents to tests
Marked as reviewed by rriggs (Reviewer).
src/java.base/share/classes/java/util/Formatter.java line 3826:
> 3824: BigDecimal tenToTheNegFour = BigDecimal.valueOf(1, 4);
> 3825: BigDecimal tenToThePrec = BigDecimal.valueOf(1, -prec);
> 3826: value = value.round(new MathContext(prec));
While you are in the area, how about inlining the creation of the tenToTheNegFour and tenToThePrec values.
They are used only once and may not be used at all. They don't appear to be needed except for the comparisons.
More information about the core-libs-dev