Release 1.7.0 to master #164
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Release 1.6.2 to dev
- a MeasurementTime has been added, to represent when the record was created. From this, MeasurementMonth and MeasurementYear have been added. - MeasurementMonth and MeasurementYear added to primary key, so that at load time we can ensure only one record (per vm) is stored, which will hopefully speed up the LRVM summariser - Alter ReplaceCloudRecord to replace old record with an incoming only if the MeasurementTime has increased. So that if messages are recieved out of order we keep the newer one (the one with the greatest MeasurementTime) - There doesn't seem to be a nicer way to do the 'ON DUPLICATE KEY UPDATE' other than field by field - And we can't get around making the greater than comparison everytime as we can only reference the current value in the 'ON DUPLICATE KEY UPDATE' block
- based on v1 as implemented by SP - no need now to compute the measurement time as this is done at load time (see proc ReplaceCloudRecord) - no need to compute the last record per month as this is done at load time as a funciton of the primary key - still need to subtract the previous months WallDuration, SuspendDuration, CpuDuration, NetworkIn/OutDuration from this months (see table TVMUsagePerMonth)
- to '00/00/00 00:00:00' - We recieve and currently accept messages without a start time which causes the mesaurementTimeCalculated to be NULL which causes a loader reject on a previously accepted message so for now, set it to the zero time stamp
- Adds Measurement fields - Sets the MeasurementTime (taking into consideration that records with a StartTime of '0000-00-00 00:00:00' cause the evaluation of MeasurementTime to be NULL - Once the MeasurementTime has been added, set MeasurementMonth and MeasurementYear - Add alter primary key to update script - Update ReplaceCloudRecord proc to choose the newer record based on measurement time - Update SummariseVMs()
- as this is the table/view that is predominantly queried for debugging - VCloudSummaries has been left unchanged at the moment, so any upgrade could be done without informing the portal
- added as it may one day be sent in the record - used in the case when a VM has completed and RecordCreateTime and MeasurementTime might be wildly different.
- remove cpuDuartionNotNull, if WallDuration is NULL we now use 0 - set suspendDurationNotNull and wallDurationNotNull regardless of which IFELSE statements are executed as these are the variables that get saved into the database, not the raw data. - Insert VO, VOGroup and VORole in the right place so FQAN is split up correctly
- rather than the notNull variables used in calculations as that is what we currently do with NULL Suspend and Wall durations
- as they are no longer needed as they are only used once, we might as well calculate it when used
- i.e. 0000-00-00 00:00:00, as this is not NULL and if the VM is running, in this case we should calculate a MeasurementTime rather than using the non-null recordCreateTime
- this means if a site contiously publish data with bad MeasurementTimes (zero timestamp) but increasing WallDuration, we at least save the increasing WallDuration (this is what would happen in production, currently under this new LRVM implementation the increasing usage it would get ignored) - also means if a site republishes with the same measurement time (unlikely but possible) the newer one would take precedent.
- we have recieved completed records with no EndTime, this causes there usage to be assigned to Month 0 Year 0. - This prevents those records being taken into consideration by the summariser. - In all the cases I have seen a valid completed message is sent eventually. So this doesnt hide any usage - Also updates the update script
- this probably slipped through before because of the lax type checking which we have now improved. - The RecordCreateTime for such records was would have been set to 'None' in the python code, rather None (or NULL in the mysql code, resulting in a load error as RecordCreateTime is non null.
Add Long Running VM support
|
I have reviewed this in #163, but GitHub seems to have gotten confused and I have simultaneously reviewed these changes and have not yet reviewed these changes.... |
|
Guess it might be to do with you being a requested reviewer. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #163 first. There should be no code in here that hasn't already been reviewed and GitHub should automatically approve this if the other one is.