New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decommission Scraper tool #718

Closed
mzeinstra opened this Issue Nov 28, 2017 · 13 comments

Comments

Projects
None yet
4 participants
@mzeinstra

mzeinstra commented Nov 28, 2017

The scraper tool that was designed to add additional metadata to a deed page from a referrer page can / needs to be scrapped (phun intended).

CC has moved to HTTPS only, which breaks this tool.

This ticket was created as a follow up on a cc developers discussion on Slack.

@paul2keller

This comment has been minimized.

Show comment
Hide comment
@paul2keller

paul2keller Nov 29, 2017

I have no idea how widespread the use of this functionality was, but I know of at least two uses in the wild. Europeana used this for all of its 20+ million PD marked objects, and I used it on a demonstration website: http://www.voyantes.net/test/pdm.html my guess is that this was not widely used but I think as a responsible steward of the licenses CC should communicate when it retires functionality related to the licenses. Since this functionality was broken as a result of the switch to https i would assume this is actually a pretty easy story to sell. Something along the lines of:

we upgraded to https only which is great because yada yada yada. Unfortunately as part of this upgrade we had to decommission the scraper tool so the display of additional metadata on our deeds functionality is not available anymore

would be great if this could still be done

paul2keller commented Nov 29, 2017

I have no idea how widespread the use of this functionality was, but I know of at least two uses in the wild. Europeana used this for all of its 20+ million PD marked objects, and I used it on a demonstration website: http://www.voyantes.net/test/pdm.html my guess is that this was not widely used but I think as a responsible steward of the licenses CC should communicate when it retires functionality related to the licenses. Since this functionality was broken as a result of the switch to https i would assume this is actually a pretty easy story to sell. Something along the lines of:

we upgraded to https only which is great because yada yada yada. Unfortunately as part of this upgrade we had to decommission the scraper tool so the display of additional metadata on our deeds functionality is not available anymore

would be great if this could still be done

@robmyers

This comment has been minimized.

Show comment
Hide comment
@robmyers

robmyers Nov 30, 2017

Contributor

Thank you for this!

That sounds good. I can make sure there's nothing that needs altering in the deed rendering code as a result (particularly anything that would affect security...). I'll talk to comms to see what we need to do but it's probably just a blog post. We should also update any references in the technical documentation on the Wiki that isn't part of a standard.

And we should dream up a future means of displaying license information on the deed. ;-)

Contributor

robmyers commented Nov 30, 2017

Thank you for this!

That sounds good. I can make sure there's nothing that needs altering in the deed rendering code as a result (particularly anything that would affect security...). I'll talk to comms to see what we need to do but it's probably just a blog post. We should also update any references in the technical documentation on the Wiki that isn't part of a standard.

And we should dream up a future means of displaying license information on the deed. ;-)

@robmyers robmyers self-assigned this Nov 30, 2017

@paul2keller

This comment has been minimized.

Show comment
Hide comment
@paul2keller

paul2keller Nov 30, 2017

thanks @robmyers for getting back on this. A blogpost noting this changes was really all I had in mind. in a perfect world scenario there would also be a change-log where CC documents all changes of the tools/licenses but I don't know if that is realistic (something like: http://rightsstatements.org/en/documentation/changelog.html)

paul2keller commented Nov 30, 2017

thanks @robmyers for getting back on this. A blogpost noting this changes was really all I had in mind. in a perfect world scenario there would also be a change-log where CC documents all changes of the tools/licenses but I don't know if that is realistic (something like: http://rightsstatements.org/en/documentation/changelog.html)

@peterspdx

This comment has been minimized.

Show comment
Hide comment
@peterspdx

peterspdx Nov 30, 2017

Contributor

+1. I was just noting that CC can do a better job documenting these same types of changes, and explaining rationales for doing so. E.g., the changes to our website, when the framing of the deeds changed, etc. Blog posts would be good for the bigger decisions, but at least a change log as Paul suggests would be great (as opposed to using Github -- something more central and available to everyone, not those just versed in Github).

Contributor

peterspdx commented Nov 30, 2017

+1. I was just noting that CC can do a better job documenting these same types of changes, and explaining rationales for doing so. E.g., the changes to our website, when the framing of the deeds changed, etc. Blog posts would be good for the bigger decisions, but at least a change log as Paul suggests would be great (as opposed to using Github -- something more central and available to everyone, not those just versed in Github).

@paul2keller

This comment has been minimized.

Show comment
Hide comment
@paul2keller

paul2keller Dec 19, 2017

@peterspdx @robmyers any news on this. We are working on a review of the Europeana data exchange agreement where we need to remove the obligation on Europeana to display the metadata use guidelines. This will be controversial and so we really need a public announcement by CC that this functionality does not exist anymore. We need tot by 10 January at the latest.

paul2keller commented Dec 19, 2017

@peterspdx @robmyers any news on this. We are working on a review of the Europeana data exchange agreement where we need to remove the obligation on Europeana to display the metadata use guidelines. This will be controversial and so we really need a public announcement by CC that this functionality does not exist anymore. We need tot by 10 January at the latest.

@peterspdx

This comment has been minimized.

Show comment
Hide comment
@peterspdx

peterspdx Dec 19, 2017

Contributor

Rob and I have a legal-tech check in today. This will be a point we'll discuss, then we'll be back to you.

Contributor

peterspdx commented Dec 19, 2017

Rob and I have a legal-tech check in today. This will be a point we'll discuss, then we'll be back to you.

@peterspdx

This comment has been minimized.

Show comment
Hide comment
@peterspdx

peterspdx Dec 19, 2017

Contributor

Rob's going to discuss further internally and will be back before the end of the year.

Contributor

peterspdx commented Dec 19, 2017

Rob's going to discuss further internally and will be back before the end of the year.

@robmyers

This comment has been minimized.

Show comment
Hide comment
@robmyers

robmyers Dec 21, 2017

Contributor

OK we are going to officially decommission this on 5th January 2018.

I will get a blog post out about this.

Contributor

robmyers commented Dec 21, 2017

OK we are going to officially decommission this on 5th January 2018.

I will get a blog post out about this.

@peterspdx

This comment has been minimized.

Show comment
Hide comment
@peterspdx

peterspdx Dec 21, 2017

Contributor

@robmyers We discussed, but it occurs to me that it's possible this breaks CC's non standard deeds. Can you please confirm that decommissioning this tool won't affect the IGO 3.0 deeds, which are almost all entirely the same but adds points that identify key differences between the IGO 3.0 licenses and the international 3.0 licenses and other ports at that version level?

Contributor

peterspdx commented Dec 21, 2017

@robmyers We discussed, but it occurs to me that it's possible this breaks CC's non standard deeds. Can you please confirm that decommissioning this tool won't affect the IGO 3.0 deeds, which are almost all entirely the same but adds points that identify key differences between the IGO 3.0 licenses and the international 3.0 licenses and other ports at that version level?

@robmyers

This comment has been minimized.

Show comment
Hide comment
@robmyers

robmyers Dec 22, 2017

Contributor

Yes I can confirm that. The "Notices" section is part of the deed text and is unaffected by the Scraper, which just adds attribution information.

Contributor

robmyers commented Dec 22, 2017

Yes I can confirm that. The "Notices" section is part of the deed text and is unaffected by the Scraper, which just adds attribution information.

@robmyers

This comment has been minimized.

Show comment
Hide comment
@robmyers

robmyers Jan 8, 2018

Contributor

After discussing with comms., I added a notice to the Wiki page, which seems to be the most official document we have for the scraper.

I also moved the scraper code to the /cc-archive/ project on GitHub.

The Scraper is no more.

Contributor

robmyers commented Jan 8, 2018

After discussing with comms., I added a notice to the Wiki page, which seems to be the most official document we have for the scraper.

I also moved the scraper code to the /cc-archive/ project on GitHub.

The Scraper is no more.

@robmyers robmyers closed this Jan 8, 2018

@paul2keller

This comment has been minimized.

Show comment
Hide comment
@paul2keller

paul2keller Jan 8, 2018

can you provide a link to the wiki page?

paul2keller commented Jan 8, 2018

can you provide a link to the wiki page?

@robmyers

This comment has been minimized.

Show comment
Hide comment
@robmyers
Contributor

robmyers commented Jan 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment