Skip to content

Commit

Permalink
Reduce rounded float precision by one more digit
Browse files Browse the repository at this point in the history
  • Loading branch information
jmuhlich committed Apr 4, 2024
1 parent cb01a97 commit f2f0cee
Show file tree
Hide file tree
Showing 2 changed files with 4,048 additions and 4,048 deletions.
8 changes: 4 additions & 4 deletions tests/lib/CsvUtils.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ static List roundColumns(String path) {
return rows
}

// Parse and round floating point values to 13 decimal places. Pass other
// Parse and round floating point values to 12 decimal places. Pass other
// strings through untouched.
static roundIfDouble(String value) {
static String roundIfDouble(String value) {
value.contains(".")
? Math.round(Double.parseDouble(value) * 1e13) / 1e13
: value
? (Math.round(Double.parseDouble(value) * 1e12) / 1e12).toString()
: value
}
Loading

0 comments on commit f2f0cee

Please sign in to comment.