Manual mesh levelling is not consistently displaying the value entered

[Thinkersbluff]

Description

This may be one of those float precision issues.

The absolute value of the data point entered is reduced by 0.001 for some entry values but not for others.

e.g. Enter 0.055, stored value displayed is 0.054.
Enter 0.050, stored value displayed is 0.050.
Enter -0.055, stored value displayed is -0.054

Steps to Reproduce

1. Power on
2. Select Level
3. Select View mesh
4. Tap any existing value
5. Try to enter the same value as a replacement
6. Most times, the absolute value of the new value is 0.001 smaller than the value entered. Some values seem to enter correctly, however.

Expected behavior:

Expect entered value to always be the stored/displayed value.

Additional Information

4.5.2 motherboard, stock TFT

CF6 Pre5 for 4.5.2 motherboard

[Provide pictures or links to videos that clearly demonstrate the issue.]

[Sebazzz]

It is actually stored correctly (verify with M503). DWIN is inconsistently truncating the floats.

[Thinkersbluff]

It is actually stored correctly (verify with M503). DWIN is inconsistently truncating the floats.

Bummer. So we cannot trust the display to confirm the entry.
I hope you are able to work around that issue. Reminds me of the problem we had with Z offset “skipping” some numbers.

is DWIN also rounding the stored values reported by Marlin and displayed in the view mesh function, or is this only affecting values we modify?

[Sebazzz]

I hope you are able to work around that issue. Reminds me of the problem we had with Z offset “skipping” some numbers.

Yes, but that was different, because I used an incorrect way to blit a floating point to the display.

is DWIN also rounding the stored values reported by Marlin and displayed in the view mesh function, or is this only affecting values we modify?

Any, actually. The input is just a whole number which we divide at the firmware side (DWIN actually doesn't support true floating point input). The values as shown on the display are true floating points, because not using floating points made it even worse 😄

[Thinkersbluff]

So if you defined the numbers as having four digits to the right of the decimal instead of three, and asked DWIN to display only the first three of those four digits, could you trick DWIN into only screwing around with the hidden digit?

[Sebazzz]

0.1811 shows as 0.181, but 0.1810 shows as 0.180. But.. 0.1800 also shows at 0.180.

That's DWIN in a nutshell. It is nuts.

[Thinkersbluff]

Rats. I guess that should just be noted as a “known issue”, in the release notes, then, if we have no way to force the display to match the value entered.

[Sebazzz]

Known issue: You have a Creality touch screen installed.

[Sebazzz]

I had one trick left up my sleeve, this seems to to the trick