Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#76 Add Sliding Window to DAGMM #77

Closed
wants to merge 3 commits into from
Closed

Conversation

WGierke
Copy link
Contributor

@WGierke WGierke commented Jun 13, 2018

This addresses #76.
Would be nice to have the name of the dataset in the title :)
[roc_syn extreme outliers pol 0 1 -2-14-2018-06-13-164021] roc_syn extreme outliers pol 0 1 -2-14-2018-06-13-164021
[roc_syn extreme outliers pol 0 3 -2-14-2018-06-13-164021] roc_syn extreme outliers pol 0 3 -2-14-2018-06-13-164021
[roc_syn extreme outliers pol 0 5 -2-14-2018-06-13-164021] roc_syn extreme outliers pol 0 5 -2-14-2018-06-13-164021
[roc_syn extreme outliers pol 1 -2-14-2018-06-13-164021] roc_syn extreme outliers pol 1 -2-14-2018-06-13-164021
[roc_synthetic combined outliers 4-dimensional-2-14-2018-06-13-164020] roc_synthetic combined outliers 4-dimensional-2-14-2018-06-13-164020
[roc_synthetic combined outliers-2-14-2018-06-13-164020] roc_synthetic combined outliers-2-14-2018-06-13-164020
[roc_synthetic extreme outliers-2-14-2018-06-13-164019] roc_synthetic extreme outliers-2-14-2018-06-13-164019
[roc_synthetic shift outliers-2-14-2018-06-13-164019] roc_synthetic shift outliers-2-14-2018-06-13-164019
[roc_synthetic trend outliers-2-14-2018-06-13-164020] roc_synthetic trend outliers-2-14-2018-06-13-164020
[roc_synthetic variance outliers-2-14-2018-06-13-164019] roc_synthetic variance outliers-2-14-2018-06-13-164019
[roc_syn variance outliers mis 0 1 -2-14-2018-06-13-164020] roc_syn variance outliers mis 0 1 -2-14-2018-06-13-164020
[roc_syn variance outliers mis 0 3 -2-14-2018-06-13-164020] roc_syn variance outliers mis 0 3 -2-14-2018-06-13-164020
[roc_syn variance outliers mis 0 5 -2-14-2018-06-13-164020] roc_syn variance outliers mis 0 5 -2-14-2018-06-13-164020
[roc_syn variance outliers mis 0 8 -2-14-2018-06-13-164021] roc_syn variance outliers mis 0 8 -2-14-2018-06-13-164021

@@ -236,7 +227,7 @@ def fit(self, X: pd.DataFrame, _):
train_energy = []
for input_data in data_loader:
input_data = to_var(input_data)
_, _, z, _ = self.dagmm(input_data)
_, _, z, _ = self.dagmm(input_data.float())
sample_energy, _ = self.dagmm.compute_energy(z, phi=train_phi, mu=train_mu, cov=train_cov,
size_average=False)
train_energy.append(sample_energy.data.cpu().numpy())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to take the mean of sample energy here as well.
I implemented it like this in the other branch:

train_length = len(data_loader)*self.batch_size + self.sequence_length - 1
train_energy = np.full((self.sequence_length, train_length), np.nan)
for i1, ts_batch in enumerate(data_loader):
    _, _, z, _ = self.dagmm(to_var(ts_batch).float())
    sample_energies, _ = self.dagmm.compute_energy(z, phi=train_phi, mu=train_mu, cov=train_cov,
                                                                                          size_average=False)
    for i2, sample_energy in enumerate(sample_energies):
        index = i1 * self.batch_size + i2
        window_elements = list(range(index, index + self.sequence_length, 1))
        train_energy[index % self.sequence_length, window_elements] = sample_energy.data.cpu().numpy()
self.train_energy = np.nanmean(train_energy, axis=0)

One issue here is that we have batches larger than 1, so we can't do it as we do in predict. In addition we're dropping the last batch, so the energy array isn't quite equal in length to data.

@danthe96
Copy link
Member

Is taken care of by #83.

@danthe96 danthe96 closed this Jun 14, 2018
@WGierke WGierke deleted the wg/dagmm_sliding_windows branch June 27, 2018 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants