Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Bioschemas by proxying Wikidata content (making Google bots happy) #1478

Merged
merged 18 commits into from Aug 31, 2021

Conversation

egonw
Copy link
Collaborator

@egonw egonw commented Jul 4, 2021

@fnielsen, this is now a finished patch, but if you have additional ideas, plz let me know (if not, please do merge in).

The new design solve the problem of the robots.txt blocking calls and limiting the SEO indexing of Scholia pages:

  • a Scholia proxy is defined with the URL pattern /$qid/bioschemas which returns JSON
  • the existing base.html uses this new call instead of a call to wikidata.org (with the robots.txt problem)

The extra call is only made when the aspect template has an id=bioschemas holder.

Possibly future optimization:

  • property_for_q() calls are replaced by a single properties_for_q(q, {"P235": "key1", "Pxx": "key2", ...})
  • 2-3 helper functions get added to simplify the code
  • other bits of the page get included in a similar way (like descriptions), making it also available for SEO
  • use the Wikidata description as Bioschemas content

@egonw egonw requested a review from fnielsen July 4, 2021 16:46
@egonw
Copy link
Collaborator Author

egonw commented Jul 4, 2021

@fnielsen, yes, I see the flake issues. I will fix those, when you told me you like the idea.

scholia/app/views.py Outdated Show resolved Hide resolved
scholia/app/views.py Outdated Show resolved Hide resolved
@fnielsen
Copy link
Collaborator

Should /$qid/bioschemas return HTML? And with content negotiation you can get json?

@egonw
Copy link
Collaborator Author

egonw commented Jul 27, 2021

Should /$qid/bioschemas return HTML?

No, the intention is only JSON.

And with content negotiation you can get json?

Yeah, I guess that would be more correct: only return JSON when asked for JSON, return 415 for any other media type.

So, go ahead?

@egonw
Copy link
Collaborator Author

egonw commented Aug 3, 2021

Okay, I will assume this is a go ahead.

@egonw egonw assigned egonw and unassigned fnielsen Aug 3, 2021
@egonw egonw added needs revision label for pull requests that are sent back for revision and removed ready for review labels Aug 3, 2021
@egonw egonw added this to the 31 August 2021 milestone Aug 3, 2021
@egonw egonw removed the request for review from fnielsen August 5, 2021 08:49
@egonw egonw changed the title Add Bioschemas by proxying Wikidata content (making Google bots happy) [WIP] Add Bioschemas by proxying Wikidata content (making Google bots happy) Aug 5, 2021
scholia/query.py Outdated Show resolved Hide resolved
egonw and others added 5 commits August 21, 2021 17:04
scholia/query.py Outdated Show resolved Hide resolved
@egonw
Copy link
Collaborator Author

egonw commented Aug 23, 2021

scholia/app/views.py Outdated Show resolved Hide resolved
@egonw egonw changed the title [WIP] Add Bioschemas by proxying Wikidata content (making Google bots happy) Add Bioschemas by proxying Wikidata content (making Google bots happy) Aug 23, 2021
@egonw egonw marked this pull request as ready for review August 23, 2021 12:24
@egonw egonw requested a review from fnielsen August 23, 2021 12:24
@egonw egonw added ready for review and removed needs revision label for pull requests that are sent back for revision labels Aug 23, 2021
@egonw egonw assigned fnielsen and unassigned egonw Aug 23, 2021
scholia/query.py Outdated Show resolved Hide resolved
scholia/query.py Outdated Show resolved Hide resolved
scholia/query.py Outdated Show resolved Hide resolved
@fnielsen
Copy link
Collaborator

Article on Bioschemas: https://scholia.toolforge.org/work/Q108345828

@fnielsen fnielsen merged commit 9d504f0 into WDscholia:master Aug 31, 2021
@fnielsen
Copy link
Collaborator

This has been reverted. It was an accidental merge and there are at least two styling errors.

@fnielsen
Copy link
Collaborator

Where is the issue for this PR?

@egonw
Copy link
Collaborator Author

egonw commented Aug 31, 2021

Where is the issue for this PR?

Basically, this is the issue: #1189

@egonw
Copy link
Collaborator Author

egonw commented Aug 31, 2021

This has been reverted. It was an accidental merge and there are at least two styling errors.

Can you give more detail, please? When I run tox -e flake8, I get:

  flake8: commands succeeded
  congratulations :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants