Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using accel(sd) affects reproducibility with random seed #187

Open
poliquin opened this issue Sep 17, 2019 · 0 comments
Open

Using accel(sd) affects reproducibility with random seed #187

poliquin opened this issue Sep 17, 2019 · 0 comments

Comments

@poliquin
Copy link
Contributor

I've noticed that when using accel(sd), changing the absorbed effects can affect the reproducibility of results even when setting a seed. Consider the following...

set seed 648290
sysuse auto, clear

gen rnd1 = runiform()
reghdfe price rnd1 mpg i.rep78, absorb(foreign)

gen rnd2 = runiform()
reghdfe price rnd2 mpg i.rep78, absorb(foreign)

set seed 648290  // start a second run of the code
loc acc "sd"

gen rnd3 = runiform()
reghdfe price rnd3 mpg, absorb(foreign rep78) accel(`acc')

gen rnd4 = runiform()
reghdfe price rnd4 mpg, absorb(foreign rep78) accel(`acc')


assert rnd1 == rnd3
assert rnd2 == rnd4  // fails when accel == sd

Perhaps this is picky, but it would be nice if I could adjust the way the model is estimated (e.g. changing absorb and accel) and still get essentially the same results on subsequent runs.

I believe the problem is line 155 from reghdfe_accelerations.mata. The implication of this line, which is actually marked BUGBUG (so maybe it's not needed?) is that changing the absorb option when using accel(sd) changes the number of times uniform gets called, which changes the fourth call to runiform above, which changes the results. That's why the first runs match, but the second one does not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant