Skip to content
This repository was archived by the owner on Nov 24, 2025. It is now read-only.

Conversation

@rob05c
Copy link
Member

@rob05c rob05c commented Jan 2, 2019

WIP - Adds Delivery Service Snapshots.

Still WIP, needs work, but I thought I'd make it available for public comment.

UPDATE 2019-11-14: I rebased to fix the conflicts, but haven't tested. Prior to the conflicts, this was AFAIK complete and working, but I still want to write a bunch of TO API Tests before removing the "WIP."

See https://cwiki.apache.org/confluence/display/TC/Traffic+Control++Self+Service+Proposal+for+Change+Integrity

This follows the plan, with one major exception: it creates new "_snapshot" tables, which are copied into on snapshot. This is because Perl touches many of these tables, and the Perl can't be easily fixed.

The idea is, if we want DS snapshots now, we can implement them much sooner, without rewriting all of Perl, with these "snapshot" tables. Then, when Perl finally goes away, we can simply create a database migration to rename the "snapshot" tables.

Specifically Includes

  • database migrations to create the "_snapshot" tables.
  • functions to copy "real" data into snapshot tables
  • a new API endpoint, deliveryservices/{ds-name}/snapshot
  • changes CRConfig Snapshot function to just insert the timestamp, and copy into snapshot tables
  • changes CRConfig Get Snapshot function to build the CRConfig on-the-fly, from the timestamp and snapshot tables
  • changes CRConfig Get Live Snapshot to copy into snapshot tables, and query their latest values (reproducing existing live behavior, without duplicate code)
  • Traffic Portal UI button to Snapshot ("Save") Delivery Service
  • monitoring.json also builds on-the-fly from snapshot tables
  • DB migration to create new snapshots for every DS on upgrade, with CDN CRConfig time
  • Add 304 Not Modified Support
  • Updated with Fix TO query for crconfig column rename #3137
  • Updated with TO fix anymap ds assignment #3317
  • Updated DB migration timestamps

update 2019-02-09
All concerns I'm aware of have been addressed. The only thing left is parity testing, to be sure everything behaves correctly.

update 2019-02-19
Need to add #3333

What does this PR do?

Which TC components are affected by this PR?

  • Documentation
  • Grove
  • Traffic Analytics
  • Traffic Monitor
  • Traffic Ops
  • Traffic Ops ORT
  • Traffic Portal
  • Traffic Router
  • Traffic Stats
  • Traffic Vault
  • Other _________

What is the best way to verify this PR?

Check all that apply

  • This PR includes tests
  • This PR includes documentation updates
  • This PR includes an update to CHANGELOG.md
  • This PR includes all required license headers
  • This PR includes a database migration (ensure that migration sequence is correct)
  • This PR fixes a serious security flaw. Read more: www.apache.org/security

@rob05c rob05c added new feature A new feature, capability or behavior Traffic Ops related to Traffic Ops crconfig-snapshot related in some way to Snapshots (CRConfig) self_service WIP "Work-in-Progress" - do not merge! (use 'draft' pull requests from now on) labels Jan 2, 2019
@asfgit
Copy link
Contributor

asfgit commented Jan 2, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3000/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Jan 22, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3083/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Jan 31, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3152/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 4, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3172/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 6, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3186/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 7, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3188/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 7, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3189/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 8, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3206/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 9, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3207/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 20, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3246/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 24, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3256/
Test FAILed.

@asfgit
Copy link
Contributor

asfgit commented Feb 25, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/3259/
Test FAILed.

Fixes CRConfig generation with DS Snapshots to return {}, like Perl.
The where query part was previously inside the last_updated sorting.
It has to be outside, because the where filtering needs to happen
after getting the latest snapshot row.

For example, if a DS is changed to inactive, putting the where
inside the 'where last_updated < snap order by last_updated'
will make the latest inactive row be excluded, and the latest
active row selected.
@asf-ci
Copy link
Contributor

asf-ci commented Nov 14, 2019

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/4782/

@zrhoffman
Copy link
Member

This should be added to the Self Service Change Integrity project because #2097 was closed in favor of it.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

crconfig-snapshot related in some way to Snapshots (CRConfig) new feature A new feature, capability or behavior self_service Traffic Ops related to Traffic Ops WIP "Work-in-Progress" - do not merge! (use 'draft' pull requests from now on)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants