Skip to content

Parsoid mod: add ability to stash wt2html transforms#363

Merged
d00rman merged 8 commits intowikimedia:masterfrom
d00rman:parsoid/stash
Oct 7, 2015
Merged

Parsoid mod: add ability to stash wt2html transforms#363
d00rman merged 8 commits intowikimedia:masterfrom
d00rman:parsoid/stash

Conversation

@d00rman
Copy link
Contributor

@d00rman d00rman commented Oct 4, 2015

This PR extends the wt2html transformations API to allow clients to indicate to RESTBase they plan to reuse the transformation at a later time, in which case RESTBase saves them in special buckets.

Summary:

  • add the stash body flag to POST /transform/wikitext/to/html
  • create the parsoid.stash.data-parsoid, parsoid.stash.wikitext and parsoid.stash.html buckets on start-up
  • save the client-sent wikitext together with the data-parsoid and html Parsoid responses for the transform when the client sets stash: true when POSTing and set the ETag to <rev>/<tid>/stash
  • detect when a client wants to use the stashed transform in a request to /transform/html/to/wikitext and reuse the stashed wikitext, data-parsoid and html originals.

Note: we should merge wikimedia/restbase-mod-table-spec#21 before merging this one.

Bug: T114548

mods/parsoid.js Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth moving this to a separate function, to avoid the stash handling obscuring the common flow too much.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking the same thing. Will do.

@gwicke
Copy link
Member

gwicke commented Oct 6, 2015

Nice! LGTM, only small tiny things inline.

mods/parsoid.js Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we have a revisionRetentionPolicy for stashed wikitext too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right

mods/parsoid.js Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be let's wait for this wikimedia/restbase-mod-table-spec#21 before it's merged?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but until then latest 0 is the best choice ;)

@d00rman d00rman changed the title [WIP] Parsoid mod: add ability to stash wt2html transforms Parsoid mod: add ability to stash wt2html transforms Oct 7, 2015
@d00rman
Copy link
Contributor Author

d00rman commented Oct 7, 2015

Ready for review, @wikimedia/services

Note 1: tests failing due to the page save API rate limit
Note 2: before merging this one we should probably resolve wikimedia/restbase-mod-table-spec#21

@Pchelolo
Copy link
Contributor

Pchelolo commented Oct 7, 2015

Could be really nice to add a couple of tests.

@d00rman
Copy link
Contributor Author

d00rman commented Oct 7, 2015

@wikimedia/services the PR is now using the ttl retention policy

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually we should be able to merge this with _getOriginalContent, but for now this is the only one loading wikitext.

@gwicke
Copy link
Member

gwicke commented Oct 7, 2015

LGTM overall, only one minor typo nit.

Marko Obrovac added 8 commits October 7, 2015 19:08
This commit introduces the stash POST body flag in
/transform/wikitext/to/html, which is used by clients to indicate they
plan to reuse the result of the transformation. The commit also
introduces the creation of the stash.data-parsoid, stash.wikitext and
stash.html buckets used to store the stashed content.
If the user sets the stash flag when POSTing to
/tranform/wikitext/to/html, the resulting data-parsoid and HTML are
stashed together with the client's wikitext.
d00rman pushed a commit that referenced this pull request Oct 7, 2015
Parsoid mod: add ability to stash wt2html transforms
@d00rman d00rman merged commit bb89e0b into wikimedia:master Oct 7, 2015
@d00rman
Copy link
Contributor Author

d00rman commented Oct 7, 2015

@Pchelolo I'll add some tests in a follow-up PR

@d00rman d00rman deleted the parsoid/stash branch October 7, 2015 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants