Very different results estimating the same model using FECT and GSYNTH

[ccepeda10]

Hello,
I noticed that I got radically different results using gsynth and fect for the same model. It's a two way model with the MC method. As the specifications of the models are identical, I was wondering what could be the cause of said difference.

Results using gsynth

Results using fect

I hope you can help me.
Cheers.

[xuyiqing]

Thank you. What if you specify the same tuning parameter lambda? It may have something to do with the different default setup for cross-validation.

…

On Wed, May 5, 2021 at 5:17 PM ccepeda10 ***@***.***> wrote: Hello, I noticed that I got radically different results using gsynth and fect for the same model. It's a two way model with the MC method. As the specifications of the models are identical, I was wondering what could be the cause of said difference. Results using gsynth [image: imagen] <https://user-images.githubusercontent.com/47113950/117224719-452b3c00-add6-11eb-8200-85a7c443eaf5.png> Results using fect [image: imagen] <https://user-images.githubusercontent.com/47113950/117224768-60964700-add6-11eb-8a55-90a7b99008bf.png> I hope you can help me. Cheers. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#16>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGF6WRIYJT3LUP55AHDTMHNYZANCNFSM44F7S2WA> .

[ccepeda10]

It's 0.1836201 in fect and 1487.01 in gsynth. That's very different. Can a high lambda be a symptom of overfit?

[xuyiqing]

Could you send me (my email) a sample dataset and code for us to replicate the results?

…

On Wed, May 5, 2021 at 5:42 PM ccepeda10 ***@***.***> wrote: It's 0.1836201 in fect and 1487.01 in gsynth. That's very different. Can a high lambda be a symptom of overfit? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#16 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2PKGB3SVLJKPHI4LA2LCLTMHQYTANCNFSM44F7S2WA> .

-- Yiqing Xu Assistant Professor Department of Political Science University of California, San Diego http://yiqingxu.org/

[xuyiqing]

I will do some tests with simulated data. On May 5, 2021, at 8:32 PM, Yiqing Xu ***@***.******@***.***>> wrote: Thank you. What if you specify the same tuning parameter lambda? It may have something to do with the different default setup for cross-validation. On Wed, May 5, 2021 at 5:17 PM ccepeda10 ***@***.******@***.***>> wrote: Hello, I noticed that I got radically different results using gsynth and fect for the same model. It's a two way model with the MC method. As the specifications of the models are identical, I was wondering what could be the cause of said difference. Results using gsynth [imagen]<https://user-images.githubusercontent.com/47113950/117224719-452b3c00-add6-11eb-8200-85a7c443eaf5.png> Results using fect [imagen]<https://user-images.githubusercontent.com/47113950/117224768-60964700-add6-11eb-8a55-90a7b99008bf.png> I hope you can help me. Cheers. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#16>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AB2PKGF6WRIYJT3LUP55AHDTMHNYZANCNFSM44F7S2WA>.

[ccepeda10]

I sent them to your e-mail. Thanks.