Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Determine If All 186 Masterfile deletes were valid #142

Closed
carrickr opened this issue Jun 15, 2017 · 11 comments
Closed

Determine If All 186 Masterfile deletes were valid #142

carrickr opened this issue Jun 15, 2017 · 11 comments

Comments

@carrickr
Copy link

We have 186 masterfiles in Avalon 4 that did not make it to Avalon 6. At least some of these are probably valid deletes and shouldn't have been migrated (old testing data that was never cleaned up, etc). However mbk and I have concerns that 186 seems rather high.

The master files are:

["numedia:10059", "numedia:10004", "numedia:10007", "numedia:10006", "numedia:10051", "numedia:10058", "numedia:10053", "numedia:10008", "numedia:10060", "numedia:10005", "numedia:10069", "numedia:10073", "numedia:10071", "numedia:10084", "numedia:10089", "numedia:10087", "numedia:10096", "numedia:10092", "numedia:10090", "numedia:10011", "numedia:10009", "numedia:10019", "numedia:10016", "numedia:10010", "numedia:10028", "numedia:10025", "numedia:10020", "numedia:10037", "numedia:10036", "numedia:10031", "numedia:10043", "numedia:10114", "numedia:9980", "numedia:9974", "numedia:9978", "numedia:9979", "numedia:9981", "numedia:9970", "numedia:9973", "numedia:11975", "numedia:11944", "numedia:11981", "numedia:11978", "numedia:11856", "numedia:11999", "numedia:11960", "numedia:11966", "numedia:11972", "numedia:11996", "numedia:11993", "numedia:11990", "numedia:11902", "numedia:11888", "numedia:12005", "numedia:12011", "numedia:11802", "numedia:9924", "numedia:9955", "numedia:9945", "numedia:9922", "numedia:9920", "numedia:9921", "numedia:9948", "numedia:9944", "numedia:9949", "numedia:9947", "numedia:9953", "numedia:9954", "numedia:9950", "numedia:9951", "numedia:9956", "numedia:9957", "numedia:9959", "numedia:9917", "numedia:9918", "numedia:9919", "numedia:11007", "numedia:11048", "numedia:11045", "numedia:11022", "numedia:11063", "numedia:11004", "numedia:11016", "numedia:11017", "numedia:11014", "numedia:11015", "numedia:11012", "numedia:11013", "numedia:11010", "numedia:11011", "numedia:11018", "numedia:11019", "numedia:11005", "numedia:11006", "numedia:11009", "numedia:11008", "numedia:11052", "numedia:11050", "numedia:11051", "numedia:11059", "numedia:11056", "numedia:11055", "numedia:11049", "numedia:11047", "numedia:11046", "numedia:11020", "numedia:11023", "numedia:11021", "numedia:11060", "numedia:23009", "numedia:14439", "numedia:14434", "numedia:14425", "numedia:14431", "numedia:14418", "numedia:15841", "numedia:26956", "numedia:47967", "numedia:20102", "numedia:20667", "numedia:20664", "numedia:26969", "numedia:26965", "numedia:26967", "numedia:21854", "numedia:23406", "numedia:24005", "numedia:24387", "numedia:24583", "numedia:24612", "numedia:24611", "numedia:24610", "numedia:47958", "numedia:3864", "numedia:26937", "numedia:26939", "numedia:26938", "numedia:26940", "numedia:26941", "numedia:26942", "numedia:26955", "numedia:26963", "numedia:28747", "numedia:28742", "numedia:28741", "numedia:28745", "numedia:28743", "numedia:28744", "numedia:28859", "numedia:28746", "numedia:28862", "numedia:46828", "numedia:47954", "numedia:47960"]
@carrickr
Copy link
Author

@mbklein

As a FYI, these 76 masterfile ids represent while Johnny Cash lost files

["numedia:10059", "numedia:10004", "numedia:10007", "numedia:10006", "numedia:10051", "numedia:10058", "numedia:10053", "numedia:10008", "numedia:10060", "numedia:10005", "numedia:10069", "numedia:10073", "numedia:10071", "numedia:10084", "numedia:10089", "numedia:10087", "numedia:10096", "numedia:10092", "numedia:10090", "numedia:10011", "numedia:10009", "numedia:10019", "numedia:10016", "numedia:10010", "numedia:10028", "numedia:10025", "numedia:10020", "numedia:10037", "numedia:10036", "numedia:10031", "numedia:10043", "numedia:10114", "numedia:9980", "numedia:9974", "numedia:9978", "numedia:9979", "numedia:9981", "numedia:9970", "numedia:9973", "numedia:11975", "numedia:11944", "numedia:11981", "numedia:11978", "numedia:11856", "numedia:11999", "numedia:11960", "numedia:11966", "numedia:11972", "numedia:11996", "numedia:11993", "numedia:11990", "numedia:11902", "numedia:11888", "numedia:12005", "numedia:12011", "numedia:11802", "numedia:9924", "numedia:9955", "numedia:9945", "numedia:9922", "numedia:9920", "numedia:9921", "numedia:9948", "numedia:9944", "numedia:9949", "numedia:9947", "numedia:9953", "numedia:9954", "numedia:9950", "numedia:9951", "numedia:9956", "numedia:9957", "numedia:9959", "numedia:9917", "numedia:9918", "numedia:9919"] 

They are present on Avalon 4, but not on AWS. So these pids can be remigrated when do our final check. Good news is this means there aren't split media objects or random unattached masterfiles running around 6 probably

I will look into the other 110 and see how many need to be remigrated.

@carrickr
Copy link
Author

carrickr commented Jun 15, 2017

Okay @mbklein

Looks like 156 of the 186 need to come across. My methodology was to pull up the masterfile on 4 and see if it had:

  • derivatives
  • a media object

If it had both, flag it as something that needs to be remigrated. The pid list is:

["numedia:10059", "numedia:10004", "numedia:10007", "numedia:10006", "numedia:10051", "numedia:10058", "numedia:10053", "numedia:10008", "numedia:10060", "numedia:10005", "numedia:10069", "numedia:10073", "numedia:10071", "numedia:10084", "numedia:10089", "numedia:10087", "numedia:10096", "numedia:10092", "numedia:10090", "numedia:10011", "numedia:10009", "numedia:10019", "numedia:10016", "numedia:10010", "numedia:10028", "numedia:10025", "numedia:10020", "numedia:10037", "numedia:10036", "numedia:10031", "numedia:10043", "numedia:10114", "numedia:9980", "numedia:9974", "numedia:9978", "numedia:9979", "numedia:9981", "numedia:9970", "numedia:9973", "numedia:11975", "numedia:11944", "numedia:11981", "numedia:11978", "numedia:11856", "numedia:11999", "numedia:11960", "numedia:11966", "numedia:11972", "numedia:11996", "numedia:11993", "numedia:11990", "numedia:11902", "numedia:11888", "numedia:12005", "numedia:12011", "numedia:11802", "numedia:9924", "numedia:9955", "numedia:9945", "numedia:9922", "numedia:9920", "numedia:9921", "numedia:9948", "numedia:9944", "numedia:9949", "numedia:9947", "numedia:9953", "numedia:9954", "numedia:9950", "numedia:9951", "numedia:9956", "numedia:9957", "numedia:9959", "numedia:9917", "numedia:9918", "numedia:9919", "numedia:11007", "numedia:11048", "numedia:11045", "numedia:11022", "numedia:11063", "numedia:11004", "numedia:11016", "numedia:11017", "numedia:11014", "numedia:11015", "numedia:11012", "numedia:11013", "numedia:11010", "numedia:11011", "numedia:11018", "numedia:11019", "numedia:11005", "numedia:11006", "numedia:11009", "numedia:11008", "numedia:11052", "numedia:11050", "numedia:11051", "numedia:11059", "numedia:11056", "numedia:11055", "numedia:11049", "numedia:11047", "numedia:11046", "numedia:11020", "numedia:11023", "numedia:11021", "numedia:11060", "numedia:23009", "numedia:14439", "numedia:14434", "numedia:14425", "numedia:14431", "numedia:14418", "numedia:15841", "numedia:26956", "numedia:47967", "numedia:20102", "numedia:20667", "numedia:20664", "numedia:26969", "numedia:26965", "numedia:26967", "numedia:21854", "numedia:23406", "numedia:24005", "numedia:24387", "numedia:24583", "numedia:24612", "numedia:24611", "numedia:24610", "numedia:47958", "numedia:3864", "numedia:26937", "numedia:26939", "numedia:26938", "numedia:26940", "numedia:26941", "numedia:26942", "numedia:26955", "numedia:26963", "numedia:28747", "numedia:28742", "numedia:28741", "numedia:28745", "numedia:28743", "numedia:28744", "numedia:28859", "numedia:28746", "numedia:28862", "numedia:46828", "numedia:47954", "numedia:47960"] 

@carrickr
Copy link
Author

We also need to do media objects numedia:47953 numedia:47959

@carrickr
Copy link
Author

carrickr commented Jun 16, 2017

@mbklein media object ids (and all masterfiles and derivatives, they've been totally cleared out of AWS Avalon)

["numedia:9916",
"numedia:11000",
"numedia:12617",
"numedia:14438",
"numedia:14432",
"numedia:14423",
"numedia:14429",
"numedia:14416",
"numedia:15755",
"numedia:17994",
"numedia:17749",
"numedia:19979",
"numedia:20661",
"numedia:22015",
"numedia:21725",
"numedia:23367",
"numedia:23966",
"numedia:24339",
"numedia:24608",
"numedia:3209",
"numedia:3795",
"numedia:26932",
"numedia:28717",
"numedia:28729",
"numedia:46827",
"numedia:47953",
"numedia:47959"]

@mbklein
Copy link

mbklein commented Jun 17, 2017

@carrickr Corresponding migrated ids of those media objects:

["2f75r8519",
"cc08hg17w",
"zg64tm70n",
"gm80hv86w",
"b5644s077",
"3t945r34g",
"8336h247h",
"bz60cw79b",
"ws859g10p",
"w3763734f",
"p8418n72b",
"8910jv059",
"ft848r10n",
"0z708x01w",
"n583xv63b",
"w3763735q",
"n009w286c",
"0z708x025",
"765371859",
"d791sg73b",
"6w924c37t",
"0z708x03f",
"w95051071",
"th83kz77f",
"jh343s81x",
"nv935341f",
"d791sg74m"]

@mbklein
Copy link

mbklein commented Jun 17, 2017

For some reason, I had to manually find and reattach two master files to one large (177 section) media object. There were also 3 strays that thought they belonged to that object – and they probably did, but were removed in 4 – that I had to get rid of. This may be the case with the other objects I migrated tonight. Several of them thought they had the wrong number of master files, but I forced them to complete with what they had. So the above 27 items will need to be hand-checked against legacy.

Please emphasize to everyone not to panic if there are small discrepancies. The master files are there; they just need to be found and wired up. This 177-section item was a pain, but it's also a good sign that it was doable.

@davidschober
Copy link

davidschober commented Jun 17, 2017 via email

@carrickr
Copy link
Author

Migration has run, since there are just 27 media objects I'll check them by hand with regard to section count and spot check individual master files for streaming

@carrickr
Copy link
Author

This is done, everything migrated, just HLS splits and attachments left.

@ccaizzi
Copy link

ccaizzi commented Jun 20, 2017

@carrickr want me to close this or leave it open/in review?

@davidschober
Copy link

@carrickr This is closed, correct? reopen if not

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants