Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENG-4377] Google dataset discovery #1842

Conversation

chth0n1x
Copy link
Contributor

@chth0n1x chth0n1x commented Apr 12, 2023

Purpose

The purpose of these changes is to return a Structured Data JSON-LD object to provide Google with a head element tag to be crawled and indexed by its bots. The aim of these changes is to improve the overall crawl rate and with the indexing and ranking of OSF resources.

Summary of Changes

-Added the returnStructuredData() method and the ScriptTagAttrs type to the Registrations Overview route to return the JSON-LD object from the osf.io/<guid>/metadata/?format=google-dataset-json-ld endpoint.
-Added ScriptTagAttrs to the exported interface and updated the Content type to include 'object' for the meta-tags service.
-Added error handling and messaging strings to the translations file when failing to return the object from the endpoint.
-Recommend: renaming the meta-tags service to the HeadTagsService to reference the script and link tags also included by the service.

Screenshot(s)

Registration - localhost

Side Effects

On different staging environments, the code will need to be re-tested to ensure the proper endpoint is being reached for the parent guid. It will fail if the flag is not set for the endpoint.

QA Notes

-Use DevTools to verify that the head element script tag displays the proper MIME type attribute as "application/ld+json" and has a valid JSON object as its content. e.g. <script type='application/ld+json' src='osf.io/...'>{jsonObject}<script/>.
-Is the script tag in its current implementation secure and efficient? Are there any attributes you would suggest adding to the tag to ensure further web application security? <async> comes to mind and a more complete list for consideration may be found here: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script.
-Mobile checks may not be necessary, but adding the view-source: prefix to the URL as in view-source:https://osf.io// would be a good check.
-Does the JSON object pass both a formatting test and this Google test? Links for each: https://jsonlint.com/ and https://search.google.com/test/rich-results. If you have any questions, please ask!

Copy link
Contributor

@adlius adlius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jotting down some of the comments made during pairing. Several major concerns:

  1. We need to make sure we are including the json-ld data in the correct route. The scope of the ticket says we should include it for registrations.
  2. The logic to fetch the json-ld data is a bit convoluted. It should be done in less 10 lines, maybe less than 3 if we don't count formatting.
  3. Adding a script-tags service solely for the purpose of setting <script> tags is redundant. We can either reuse the meta-tags service or use the head-tags service directly.

app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/services/script-tags.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/services/script-tags.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
app/services/script-tags.ts Outdated Show resolved Hide resolved
translations/en-us.yml Outdated Show resolved Hide resolved
@chth0n1x chth0n1x changed the title [DRAFT][ENG-4377] Google dataset discovery [ENG-4377] Google dataset discovery Apr 26, 2023
@coveralls
Copy link

coveralls commented Apr 26, 2023

Pull Request Test Coverage Report for Build 5189939723

  • 10 of 13 (76.92%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.02%) to 72.693%

Changes Missing Coverage Covered Lines Changed/Added Lines %
lib/registries/addon/overview/route.ts 10 13 76.92%
Totals Coverage Status
Change from base Build 4852574086: -0.02%
Covered Lines: 5738
Relevant Lines: 7682

💛 - Coveralls

app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
app/guid-file/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
app/services/script-tags.ts Outdated Show resolved Hide resolved
app/services/script-tags.ts Outdated Show resolved Hide resolved
app/services/script-tags.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
@chth0n1x chth0n1x force-pushed the feature/google-dataset-discovery branch 3 times, most recently from 064d9ff to cf61611 Compare May 31, 2023 17:59
app/services/meta-tags.ts Outdated Show resolved Hide resolved
app/services/meta-tags.ts Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
translations/en-us.yml Outdated Show resolved Hide resolved
app/services/meta-tags.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
app/services/meta-tags.ts Outdated Show resolved Hide resolved
app/services/meta-tags.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
lib/registries/addon/overview/route.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@futa-ikeda futa-ikeda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One subtle thing I see is that in overview/route.ts, we are setting
this.headTags = this.metaTags.getHeadTags(metaTagsData); in line 83,
and then pushing to this.headTags in line 96 and line 104. I've definitely run into issues before when adding/removing elements to an array and things that depend/read-from the array not updating properly. Since headTags isn't tracked or an EmberArray, I don't know if the tags pushed in line 96 and 104 are going to be included just by looking at this code, so I would double-check and make sure those are indeed visible in the <head>.

The previous implementation was adding all the tags to the variable const headTags and then doing this.set('headTags', headTags), so this may be the safest way of implementing the headTags to make sure things aren't lost.

adlius
adlius previously approved these changes Jun 6, 2023
@chth0n1x
Copy link
Contributor Author

chth0n1x commented Jun 6, 2023

One subtle thing I see is that in overview/route.ts, we are setting this.headTags = this.metaTags.getHeadTags(metaTagsData); in line 83, and then pushing to this.headTags in line 96 and line 104. I've definitely run into issues before when adding/removing elements to an array and things that depend/read-from the array not updating properly. Since headTags isn't tracked or an EmberArray, I don't know if the tags pushed in line 96 and 104 are going to be included just by looking at this code, so I would double-check and make sure those are indeed visible in the <head>.

The previous implementation was adding all the tags to the variable const headTags and then doing this.set('headTags', headTags), so this may be the safest way of implementing the headTags to make sure things aren't lost.

Ok, that sounds good. The goal was to reduce the total number of lines but I agree. I can add a separate placeholder for the headTags value to push to before setting this.headTags.

@adlius adlius changed the base branch from develop to basket/cesium June 7, 2023 03:52
@futa-ikeda futa-ikeda merged commit ee97a8f into CenterForOpenScience:basket/cesium Jun 13, 2023
@brianjgeiger brianjgeiger added this to the 23.08.0 milestone Jul 13, 2023
@chth0n1x chth0n1x deleted the feature/google-dataset-discovery branch October 20, 2023 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants