Affine transforms - matrix algebra: equals

Kirill.Prazdnikov kirill.prazdnikov at
Thu Aug 23 14:15:39 PDT 2012

On 8/24/2012 12:52 AM, Jim Graham wrote:
> In other words, you'd specify "within 1/Nth of a pixel", not "within N 
> bits of mantissa" and if you are measuring over the dimensions of a 
> typical screen (0-2K for example) then "1/Nth of a pixel" is around 12 
> bits of mantissa for N=1 (or ~(212)/N multiple of ulp) for a "float", 
> and around 40 bits of mantissa for N=1 for a "double".  This form of 
> the test would probably do better with an absolute error measurement. 

You are thinking as if everything is screen space 2D. But when we see a 
space ship flying around then the math above is different ... ( + camera 
transformation errors )

ULP-precision is the absolute measurement of precision. Despite of 


More information about the openjfx-dev mailing list