Skip to content

Fix truncation of coordinates #354

Description

@briantoby

When a reading a coordinate from a text file there will be truncation and roundoff for fractional coordinates where n/m is not m is not 2,4,8,... We could convert coordinates to the exact fraction (to double precision accuracy) when a number is close.

The test could be something like this:

x12 = 12*x
ix12 = closest integer to x12
if abs(x12-ix12) < tol then make x = ix12/12

It would be best done for all phase imports from text files (which is all phase imports at present?) and tol could even be set by the number of decimal places in the file.

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions