Float printf, floor and float2nr differ

let p = 0.705882352941176
let pf = p * 255
echo pf
echo printf("%.10f",pf)
echo float2nr(pf)
echo floor(pf)

180.0
180.0000000000
179
179.0

I have a Vim script app that converts between RGB numbers
and HSV floats. One would expect that going from RGB to
HSV and back to RGB would result in the original RGB values
but both floor and float2nr produce rounding errors.
If I add 0.000000000001 to values in the HSV converter
just prior to applying float2nr, I get no rounding errors.

Of course, using printf to convert the float to a string,
removing the decimal point character and all characters after
it in the string, and then converting that string into a
number also works. But, is rather heavy.


Richard

--
Quis custodiet ipsos custodes

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php