Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong CALT outputs for sample size 512 and 1000 #303

Closed
johcarter opened this issue Jun 14, 2022 · 1 comment · Fixed by #305
Closed

Wrong CALT outputs for sample size 512 and 1000 #303

johcarter opened this issue Jun 14, 2022 · 1 comment · Fixed by #305
Assignees
Labels
Milestone

Comments

@johcarter
Copy link
Contributor

the estimates for sample size 512 and 1000 are wrong (very large ) for piwind @hchagani-oasislmf
to replicate run piwind with 1000 samples and view gul_S1_alct report
please can you take a look when you get back? cheers

Originally posted by @johcarter in #294 (comment)

@johcarter johcarter added the bug label Jun 21, 2022
@hchagani-oasislmf
Copy link
Contributor

I was not able to recreate the very large numbers you saw @johcarter. However, a sample subset size of 512 should not have been possible given a total sample size of 1000, as the samples in each subset are non-overlapping. Therefore, a subset size of 512 would range from sample 512 to (512 + 512 - 1) = 1023. This is 23 samples greater than the total sample size, so accessing these samples as elements in the vector of losses can return junk. In your case, it may have returned very large values, thus skewing the estimates. In my case, it may have returned very low values, thus making little difference.

Regardless, the behaviour is incorrect. I have fixed this, and this should drop the 512 samples subset and give you the correct estimates for 1000 samples.

@hchagani-oasislmf hchagani-oasislmf self-assigned this Jun 28, 2022
@awsbuild awsbuild added this to the v3.9.1 milestone Jul 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants