-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Different WAPE compare to paper #3
Comments
Hi @baghishani, |
Hi @joppichristian, thanks for your response. Both train and evaluation is for 6 weeks. |
We trained the model on 12 weeks and test on 6. |
Thanks a lot. I will try that :) |
Dear @joppichristian, I trained the model on 12 weeks and then tested it on [12,8,6,4] weeks, and here are my results: Do you think is there any difference between the train I did and yours, since I got different results? |
There might be some differences in the parameters and the initialization of the network (it depends on the hardware and other variables). I am unsure about the parameters; they should be those reported in the scripts as default. |
Dear HumaticsLAB,
First of all, thanks for sharing your code.
I ran this model with all default parameters except instead of 12 weeks for 6 weeks and for model I chose "cross".
The WAPE I got is 53.46 while the WAPE in GTM paper for Cross model with Gtrend is 59.0.
I used your formula for WAPE in code. I highlighted the row in this picture
![GTM](https://private-user-images.githubusercontent.com/32515686/324808127-8409e247-5774-4265-ba5b-ef2ae982caab.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjE1MDMxMjUsIm5iZiI6MTcyMTUwMjgyNSwicGF0aCI6Ii8zMjUxNTY4Ni8zMjQ4MDgxMjctODQwOWUyNDctNTc3NC00MjY1LWJhNWItZWYyYWU5ODJjYWFiLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MjAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzIwVDE5MTM0NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTRlM2Q2NTliN2FlNjFmNGU3YWM4MDBjYTNiNzIyOWUxNGY2NTI4ODhjMzg1Y2NhM2ZhNDJiY2IxMGViMDg5MDImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.cZqu3782bC_G6zhbh_bjsXjsYLppeReAjhfhDoFAmQY)
In the last four rows of this image, i got the same values by running models, only cross with Gtrend is different.
Could you please help me? I need to know how produce the same scores as you did for my comparison part as base-line models.
Bests,
Maryam
The text was updated successfully, but these errors were encountered: