Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hello, thank you for your implementation. I don't know if it's abnormal or not. #11

Open
waystogetthere opened this issue May 14, 2021 · 2 comments

Comments

@waystogetthere
Copy link

Hello, thank you for your implementation. It is well written and easy to go through.

I downloaded the code and ran the synthetic hawkes data set under the default setting I found that the predicted inter-event time remained the same.
Figure_1

This plot is the mean predicted inter-event time and mean ground truth inter-event time for 64 sequences in hawkes data set.

Also, the training loss does not change significantly during the training.

Starting epoch... 0
2021-05-14 11:16:23.015432: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10
Loss during batch 0 last BPTT = -4.655, lr = 0.00999
Loss on last epoch = -4.6547, new lr = 0.00999, global_step = 70
Starting epoch... 1
Loss during batch 0 last BPTT = -6.581, lr = 0.00999
Loss on last epoch = -6.5805, new lr = 0.00999, global_step = 140
Starting epoch... 2
Loss during batch 0 last BPTT = -6.584, lr = 0.00998
Loss on last epoch = -6.5837, new lr = 0.00998, global_step = 210
Starting epoch... 3
Loss during batch 0 last BPTT = -6.585, lr = 0.00997
Loss on last epoch = -6.5846, new lr = 0.00997, global_step = 280
Starting epoch... 4
Loss during batch 0 last BPTT = -6.585, lr = 0.00997
Loss on last epoch = -6.5855, new lr = 0.00997, global_step = 350
Starting epoch... 5
Loss during batch 0 last BPTT = -6.586, lr = 0.00996
Loss on last epoch = -6.5863, new lr = 0.00996, global_step = 420
Starting epoch... 6
Loss during batch 0 last BPTT = -6.587, lr = 0.00995
Loss on last epoch = -6.5871, new lr = 0.00995, global_step = 490
Starting epoch... 7
Loss during batch 0 last BPTT = -6.588, lr = 0.00994
Loss on last epoch = -6.5879, new lr = 0.00994, global_step = 560
Starting epoch... 8
Loss during batch 0 last BPTT = -6.589, lr = 0.00994
Loss on last epoch = -6.5887, new lr = 0.00994, global_step = 630
Starting epoch... 9
Loss during batch 0 last BPTT = -6.590, lr = 0.00993
Loss on last epoch = -6.5895, new lr = 0.00993, global_step = 700
Starting epoch... 10
Loss during batch 0 last BPTT = -6.590, lr = 0.00992
Loss on last epoch = -6.5903, new lr = 0.00992, global_step = 770

I don't know if it's abnormal or not. Any reply will be appreciated.
Thank you very much!

@musically-ut
Copy link
Owner

That is ... neither here nor there. I cannot say off the top of my head whether that is a good prediction or not, it seems to be the mean of the durations.

Does the same behavior persist with other example tests as well?

@waystogetthere
Copy link
Author

Hello! Sorry for being away for a while. This week I am a bit overwhelmed for some deadlines.
The RMTPP is a great work attracting continuous discussion. I am quite inspired and Thank you for your elegant work!


Yes the green line is the mean predicted duration at each event-stamp for all test cases.

pred_dur = test_time_preds - data['test_time_in_seq'][:, :test_time_preds.shape[1]]

And the red line indicates the ground truth mean duration at each event-stamp for all test cases.

gt_dur = data['test_time_out_seq'] - data['test_time_in_seq']

The graph is plot in a way:

plt.plot(np.mean(pred_dur, axis=0), label='Predicted Duration', color='green')
plt.plot(np.mean(gt_dur,` axis=0), label='Ground Truth Duration', color='red')

However, I think this is not a good prediction as at each event-stamp it predicts the same duration.
This is a mean plot and I manage to extract some predictions for specific test cases:
Figure_1

I found that in your code the MAE is chosen as a metric to evaluate the model's time prediction performance. I may be more interested in the interval time vs time index plot, as the paper shows in figure 4. https://www.kdd.org/kdd2016/papers/files/rpp1081-duA.pdf

Thank you, Wish you a good day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants