Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix 4459 #5586

Closed
wants to merge 42 commits into from
Closed

Bugfix 4459 #5586

wants to merge 42 commits into from

Conversation

arlobelshee
Copy link

@arlobelshee arlobelshee commented May 29, 2018

Resolves bug #4459 by, when an excerpt is present on the incoming node, parsing it exactly like the node's content.

Implemented this by extracting the functions involved in the parsing from the caching behavior around them, then calling them twice (once for each part). The lowest-level function (which actually does the remark parsing to an AST, asynchronously) is triggered on both parts as soon as the code asks for either. It parses the two in parallel, storing the pair of results in the cache.

The outer functions (getHTML() and friends) are simply duplicated, with the excerpt HTML stored in the cache just like the content HTML. By duplicated, I mean extracted, parameterized by wrapper, and then the wrapper is duplicated for the other behavior. This minimizes code duplication.

Note on my commit messages: each commit is very small. The first character indicates my attempted risk level / kind of transform for that commit. I did make a couple errors along the way; I think I caught them.

The risk level codes I use are:

  • r = disciplined refactoring. Low risk.
  • a = autoformat done by a tool. Low risk.
  • !!! = my intent was to refactor, but I don't have a recipe for this, so it is much riskier.
  • F = work on the feature. Intentionally changes behavior, so there is more risk.
  • B = bugfix. I made a mistake, caught it, and fixed it. Refers to where the mistake was made. Bugfixes are inherently risky themselves.

Finally, I am totally unfamiliar with this code and have no idea how good your test coverage is. I did not extend it. So I believe everything works, but I can't know how confident to be. Some contributor, please help verify!

… the async nature. I think I got it right, but good to check this one).
…r that when I duplicated the call to generate its AST. Thus, that introduced a bug. Now fixed.
…oke when lifting the function out of its scope.
@gatsbybot
Copy link
Collaborator

gatsbybot commented May 29, 2018

Deploy preview for using-drupal ready!

Built with commit fc55e6b

https://deploy-preview-5586--using-drupal.netlify.com

@gatsbybot
Copy link
Collaborator

gatsbybot commented May 29, 2018

Deploy preview for gatsbygram ready!

Built with commit fc55e6b

https://deploy-preview-5586--gatsbygram.netlify.com

…. Bug introduced when I lifted the function out one scope.
…liar with this project and don't know all conventions.
… be parsed in some cases and perhaps not in others.
…the digest hash. Updated the test, but this is risky - could be an unexpected side effect bug?
…what they do with it. So they are really a test of greymatter, not of this code. Updated snaps to show that they don't parse markdown in excerpt or content.
…ult for the digest hash. Updated the test, but this is risky - could be an unexpected side effect bug?"

This reverts commit 977dc3d.
@arlobelshee
Copy link
Author

I reviewed test coverage. There is 0 coverage of the function I modified. It isn't even imported or executed by tests.

I tried to add coverage, but the function takes 6 params, many of which are complicated objects. I couldn't figure out how to get either real or fake ones, so I abandoned it. Hopefully a contributor can figure out how to get the params to call this method!

@oriSomething
Copy link

Does someone from the maintainer can please look on this PR?
This PR is opened for quite a long time, and for me and I guess many other can solve the excerpt issue.

@pieh
Copy link
Contributor

pieh commented Sep 20, 2018

Could we do it other way - ie in here were we set excerpt from graymatter

markdownNode.excerpt = data.excerpt

just strip markdown (for example using https://www.npmjs.com/package/remove-markdown)?

---edit: don't mind me I missunderstood this

* master: (1971 commits)
  Delete cds-takeaways.md (#8552)
  chore(release): Publish
  chore: remove npm-run-all dep and update deps in e2e tests (#8529)
  [www] Site/Starter Showcase refactor, rinse II (#8411)
  Rfc process doc (#8537)
  [www] a11y improvements (#8536)
  Update the GraphQL stitching blog post & docs (#8516)
  fix: change teams to reduce noise for people
  Category typos fixes and consolidation (#8520)
  debugging build: error due to import/require mix (#8493)
  updating name
  Adding pagination
  docs: add pagination doc (#8476)
  test: add more unittests for transformer remark (#8272)
  chore: simplify editorconfig (#8412)
  chore: format YAML files with Prettier (#8407)
  feat: add noInlineHighlight option (#7554)
  chore: bump drupal plugin major version - gatsby v1 version of plugin already uses v2.x.x (#8499)
  Added option to minify css class names generated in production build for react-css-modules. (#8496)
  chore: add code owners for www and critical plugins (#8518)
  ...
@m-allanson
Copy link
Contributor

I've just merged master into this branch, and this seems to be working. Thanks @arlobelshee for the fix 🙏

I want to do some more testing on this before merging, which I'm aiming to do later this week. @pieh mentioned that there could be performance impact with generating two AST's, so there may be further updates needed there.

@m-allanson
Copy link
Contributor

Also thanks to @oorestisime's work (🎉) on the unit tests for gatsby-transformer-remark I should be able to update this PR with specific unit tests too.

}
const futureContent = processMarkdown(markdownNode.internal.content)
const futureExcerpt = markdownNode.excerpt
? processMarkdown(markdownNode.excerpt)
Copy link
Contributor

@pieh pieh Oct 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will process excerpt markdown even if it won't be used - it would be better if this would be more lazy and only process markdown (both for excerpt and content) when someone is querying for it. It would need separate cache keys for content and excerpt ASTs

@LekoArts
Copy link
Contributor

LekoArts commented Dec 4, 2018

@pieh Is this PR still relevant seeing that #9716 was merged?

@pieh
Copy link
Contributor

pieh commented Dec 4, 2018

Yes, we can close this one. Thanks for cleanup session @LekoArts!

@pieh pieh closed this Dec 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants