Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stat produces incorrect results when using secondary table #12

Closed
snystrom opened this issue Nov 30, 2020 · 8 comments
Closed

stat produces incorrect results when using secondary table #12

snystrom opened this issue Nov 30, 2020 · 8 comments

Comments

@snystrom
Copy link

I'm having an error where stat produces incorrect results depending on how the d4 file was generated.

If generated using the default d4tools create, I get incorrect values for mean and median. However, if generating the file using d4tools create -S, that d4 file produces accurate median and stat measurements. Accessing the values using d4tools view is accurate, however for both types of d4 file, and if I compute a rough estimate of the average from the view scores (without accounting for binning), I get a correct-ish number in both instances.

My computed stats using d4tools view:
Default D4:
- min: 255
- mean (approximate): 273.769
Sparse D4:
- min: 255
- mean (approximate): 273.769

d4tools stats:
default compression
- mean: 63
sparse compression
- mean: 273.5447154471545
@snystrom
Copy link
Author

snystrom commented Nov 30, 2020

And a reproducible example using d4tools 0.1.16:
issue-12-example.tar.gz

md5sum: 4dbe38988bb191fe093336c3d83d6961 issue-12-example.tar.gz

@snystrom
Copy link
Author

Not to be a nuisance, but is this a case of user error, or is this a real bug? This effect is a major blocker for adoption of this format for a few projects.

@arq5x
Copy link
Collaborator

arq5x commented Aug 18, 2021

I agree, @38 could you look into this ASAP?

@arq5x
Copy link
Collaborator

arq5x commented Aug 18, 2021

@snystrom just to confirm, this behavior exists on the latest (0.2.18) version as well, correct?

@snystrom
Copy link
Author

Yeah I just checked it.

38 added a commit that referenced this issue Aug 18, 2021
@38
Copy link
Owner

38 commented Aug 18, 2021

Hi there, thanks for reporting this issue. It should be fixed by the latest commit. It would be great if you can confirm that. Thanks!

@arq5x
Copy link
Collaborator

arq5x commented Aug 18, 2021

Great. This fixed it on my laptop.

@snystrom
Copy link
Author

Confirm also fixed on my end, thanks a bunch!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants