Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Number of significant digits #5

Closed
DieterLa opened this issue Aug 29, 2023 · 3 comments
Closed

Number of significant digits #5

DieterLa opened this issue Aug 29, 2023 · 3 comments
Assignees
Labels
bug Something isn't working enhancement New feature or request

Comments

@DieterLa
Copy link

Results are shown with more digits than what is really relevant.
e.g.
Ee,lc 807.000 mW/m²
correct would be 807.312
maybe this comes because my data is in mW/nm, therefore the results are magnified by a factor of 1000.
But this looks like data is rounded to full W/m²

@DieterLa
Copy link
Author

For the M-cone, the result is 719,8, but is rounded to 720. This is not nice.
Accuracy should be better.

@JZauner
Copy link
Collaborator

JZauner commented Sep 15, 2023

So, this covers several topics.

  • One is importing. As I said in Decide and inform user about units for data import mW/nm or W/nm #4 , the data is supposed to be in W/m²*nm and needs to be scaled within the import options if it is not. So your data will be 807 W/m² instead of mW/m² if you did not scale it. Giving the option as part of Decide and inform user about units for data import mW/nm or W/nm #4 will hopefully make this clearer

  • One is wanted rounding. What it does is show 0 decimals for values above 100, 1 decimal for values between 10 and 100, 2 decimals between 1 and 10, and 3 decimals below 1. Looking through many spectral measurements this seemed like a good balance between exact values and nice table formatting. I realize that I need to let people know of this rounding schema, and ideally let them opt out of it to get more exact numbers. You can already get the unrounded values through the Excel-Export.

  • One is unwanted rounding, so a bug :). It rounds to three significant values, which is very much undesirable above values of 1000, so I need to fix this.

@JZauner JZauner self-assigned this Sep 15, 2023
@JZauner JZauner added bug Something isn't working enhancement New feature or request labels Sep 15, 2023
@JZauner
Copy link
Collaborator

JZauner commented Nov 3, 2023

Fixed the bug. It will go online in Version 1.0.0 of Spectran

@JZauner JZauner closed this as completed Nov 3, 2023
JZauner added a commit that referenced this issue Nov 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants