Add previous values to app usage events

[hsiliev]

CC presents the app usage events in chronological order, but this doesn't guarantee that they'll progress in that order through micro-service pipeline (such as Abacus). There may be random communication delays, errors, retries, etc along the processing of each event, which may disturb the initial chronological order.

This PR tries to solve these out-of-order submissions by using special formulas based on previous state as described in cloudfoundry-attic/cf-abacus#231

[cfdreddbot]

Hey hsiliev!

Thanks for submitting this pull request! I'm here to inform the recipients of the pull request that you've already signed the CLA.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/116521193.

[ameowlia]

This has been merged. I don't know why it didn't automatically update this PR.

Thanks for the well tested PR @hsiliev,
Amelia, Member of the CAPI Team

[hsiliev]

Nor there was any notification.

Thanks a lot for merging the PR

[SocalNick]

Hi @hsiliev,

Do we have any sense how long the migration would take for large deployments with millions or tens of millions of app usage events? I think each add_column call is an alter table which requires a temp table, at least in MySQL.

Thanks,

Nick

[hsiliev]

Hi Nick,

Nope - we only have a bit over half a million events, and we didn't notice
problems with the migration. But you are right, this might take quite a
while for busy installations.

I noticed that we already had such migration in
cloud_controller_ng/db/migrations/20160303231742_add_package_guid_to_usage_events.rb,
so I decided this went ok. However we now have 4 new columns.

Regards,
Hristo Iliev

2016-04-06 2:46 GMT+03:00 Nicholas Calugar notifications@github.com:

Hi @hsiliev https://github.com/hsiliev,

Do we have any sense how long the migration would take for large
deployments with millions or tens of millions of app usage events? I think
each add_column calls is an alter table which requires a temp table, at
least in MySQL.

Thanks,

Nick

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#569 (comment)