Why does the normalization apply only to the input?

[yaoweihu]

Hello, I see your code, and I find that you only apply log and normalization to the in_times? I don't understand why not apply them to in_times and out_times?

[shchur]

Since we are using log-likelihood as the evaluation metric, applying transformations to inter-event times (e.g. scaling / logartihm) will change the results. The log-likelihood is defined as \sum_i \log p^*(\tau_i), where p^*(\tau_i) is the conditional density at point \tau_i - we are summing log-densities for all the samples in the dataset. If we transform all the inter-event times \tau_i (e.g. scale / apply log), the densities will also get changed according to the change of variables formula.

Instead, we do the following. All the models considered in our paper are defined as normalizing flows (i.e. a sequence of transformations of a base density).

z_0 ~ p(z_0)
z_1 = g_1(z_0)
...
z_M = g_M(z_{M-1})
tau = f(z_M)

As the final transformation (f in the pseudocode above) we apply scaling / exponentiation (see decoders.py). This way

1. we obtain a distribution over original inter-event times
2. the models are easier to train, since the distribution over z_M should have 0 mean and unit variance.

Let me know if this explanation doesn't make sense to you and I will try to clarify it.