Improved server rendering #9343
Conversation
|
|
||
| #### Meteor Accounts | ||
| Meteor's authentication system uses cookies to store the login token for | ||
| an authenticated user. To get access to this, you can use the `getCookies` method |
benjamn
Nov 12, 2017
Member
I wish this statement was true, since it would allow server-side rendering to work much better with with the Meteor accounts-* packages. Unfortunately, the login token is stored in the browser's localStorage, which means it can't be sent in a cookie, and must be read by client JavaScript after the response is sent back.
I wish this statement was true, since it would allow server-side rendering to work much better with with the Meteor accounts-* packages. Unfortunately, the login token is stored in the browser's localStorage, which means it can't be sent in a cookie, and must be read by client JavaScript after the response is sent back.
|
|
||
|
|
||
|
|
||
| #### React 16 renderToNodeStream |
benjamn
Nov 12, 2017
Member
Let's put markdown backticks around this function?
Let's put markdown backticks around this function?
|
|
||
| // server only methods | ||
| setStatusCode() { | ||
| console.error(isoError("setStatusCode")); |
benjamn
Nov 12, 2017
Member
Could just move the console.error call into isoError.
Could just move the console.error call into isoError.
|
|
||
| const end = this.closeTemplate(data); | ||
|
|
||
| return { start, stream, end } |
benjamn
Nov 12, 2017
Member
Can't we just include the start and end chunks in the stream, so we don't have to return multiple things here?
Can't we just include the start and end chunks in the stream, so we don't have to return multiple things here?
jbaxleyiii
Nov 15, 2017
Author
Contributor
haha duh, wow should've done that from the start (see what I did there 😉 )
haha duh, wow should've done that from the start (see what I did there
| stream.on("end", () => { | ||
| res.write(end); | ||
| res.end(); | ||
| }) |
benjamn
Nov 12, 2017
Member
For example, if start and end were part of the stream, this code could just be
stream.pipe(res, { end: true });
For example, if start and end were part of the stream, this code could just be
stream.pipe(res, { end: true });| @@ -8,6 +8,24 @@ var hash = crypto.createHash('sha1'); | |||
| hash.update(additionalScript); | |||
| var additionalScriptPathname = hash.digest('hex') + ".js"; | |||
|
|
|||
| // convert a stream to a string via promise | |||
| function toString(stream) { | |||
benjamn
Nov 12, 2017
Member
Alright, this code is duplicated three times now. Let's find a better home for it!
Alright, this code is duplicated three times now. Let's find a better home for it!
| var string = '' | ||
| stream.on('data', function(data) { | ||
| string += data.toString(); | ||
| }); |
benjamn
Nov 12, 2017
Member
I think it might be better if we appended the data buffers to an array and then used Buffer.concat(chunks).toString("utf8") at the end, since a chunk boundary could theoretically split a multi-byte unicode character.
I think it might be better if we appended the data buffers to an array and then used Buffer.concat(chunks).toString("utf8") at the end, since a chunk boundary could theoretically split a multi-byte unicode character.
| @@ -3,6 +3,10 @@ Package.describe({ | |||
| version: '1.3.1' | |||
| }); | |||
|
|
|||
| Npm.depends({ | |||
| "combine-streams": "1.0.0" | |||
benjamn
Nov 12, 2017
Member
I worry that the combine-streams package is not very heavily developed/starred/downloaded. In particular, it doesn't seem to pay any attention to the string encoding of the written chunks.
After a bit of digging, this combined-stream2 package looks a bit more mature.
I worry that the combine-streams package is not very heavily developed/starred/downloaded. In particular, it doesn't seem to pay any attention to the string encoding of the written chunks.
After a bit of digging, this combined-stream2 package looks a bit more mature.
|
@benjamn thanks for the comments! I'll work to address them ASAP! |
|
Just wanted to say it's awesome to see things moving so fast! Hope to be able to integrate this in Vulcan soon :) |
|
@jbaxleyiii Since hack week is over and you're back busy on all things Apollo, just say the word and I'll jump in to finish this PR up. I'm super excited to get these changes out - thanks for working on this! |
|
@hwillson if you have time that would be amazing! There are a number of apollo client 2.0 upgrade blockers that I need to put my full focus on. I'm going to try to get all of the changes done by Friday but if not then if you and @benjamn can take it over it would be so great to see this one make it through. @benjamn does that timeline work? I want to make sure its ready for PR club next week |
|
Next week is great, @jbaxleyiii. Thanks for your work on this! |
jbaxleyiii
force-pushed the
jbaxleyiii/streaming-server-render
branch
from
8a43f71
to
8e6a6c6
|
@benjamn of careful note is the removal of the boilerplate memoization (done on a separate commit to make it easy to revert) I think I'm probably misunderstanding its goals, but given the potential dynamic nature of generating markup within an app, memoization of request => markup seems like a bad idea? |
|
@hwillson one thing this is still missing is tests for redirection, setting status codes, and getting / setting headers. I don't think I'll have time to write those sadly :( |
| @@ -50,5 +52,7 @@ export function generateHTMLForArch(arch) { | |||
| }, | |||
| }); | |||
|
|
|||
| return boilerplate.toHTML(); | |||
|
|
|||
| return await toString(boilerplate.toHTML()); | |||
benjamn
Nov 20, 2017
Member
No need to explicitly await when returning a value that could be a Promise from an async function.
No need to explicitly await when returning a value that could be a Promise from an async function.
| /<script[^<>]*>[^<>]*__meteor_runtime_config__ =.*decodeURIComponent\(config123\)/ | ||
| ); | ||
| } | ||
| run().then(onComplete); |
benjamn
Nov 20, 2017
Member
I think you need to handle the error case here, too, so the test doesn't time out if there's an error.
I think you need to handle the error case here, too, so the test doesn't time out if there's an error.
| } | ||
| ); | ||
| const start = async () => { | ||
| const html = await generateHTMLForArch('web.browser'); |
benjamn
Nov 20, 2017
Member
Rather than using an async function here, I would just do
generateHTMLForArch("web.browser").then(html => {
Tinytest.add(...);
...
});
I'm not entirely sure what happens if the tests finish while there's still a pending Promise, so you might want to capture the resulting promise and add an additional
Tinytest.addAsync("everything ok", function (test, onComplete) {
promise.then(
() => onComplete,
error => test.fail(error)
);
});
at the end of the file.
Rather than using an async function here, I would just do
generateHTMLForArch("web.browser").then(html => {
Tinytest.add(...);
...
});I'm not entirely sure what happens if the tests finish while there's still a pending Promise, so you might want to capture the resulting promise and add an additional
Tinytest.addAsync("everything ok", function (test, onComplete) {
promise.then(
() => onComplete,
error => test.fail(error)
);
});at the end of the file.
benjamn
Nov 20, 2017
Member
We should really just improve Tinytest to handle Promises better, honestly, but that's a mission for a different PR.
We should really just improve Tinytest to handle Promises better, honestly, but that's a mission for a different PR.
| const end = this.closeTemplate(data); | ||
| const response = stream.create(); | ||
|
|
||
| response.append(Buffer.from(start)) |
benjamn
Nov 20, 2017
Member
Let's be explicit about passing "utf8" as the encoding type as the second argument to Buffer.from.
Let's be explicit about passing "utf8" as the encoding type as the second argument to Buffer.from.
|
|
||
| response.append(Buffer.from(start)) | ||
| if (body) { | ||
| response.append(typeof body === "string" ? Buffer.from(body) : body) |
benjamn
Nov 20, 2017
Member
What else can body be? Let's enforce our assumptions here using some combination of typeof body === "string", Buffer.isBuffer, and typeof body.read === "function" (if that covers all the cases).
What else can body be? Let's enforce our assumptions here using some combination of typeof body === "string", Buffer.isBuffer, and typeof body.read === "function" (if that covers all the cases).
benjamn
Nov 20, 2017
Member
The internet tells me body instanceof ReadableStream isn't always safe, since folks can implement their own stream-like objects, which is why I suggested the dynamic duck test that body.read is a function. Maybe extract this logic into a helper function so you can do appendToStream(response, body)?
The internet tells me body instanceof ReadableStream isn't always safe, since folks can implement their own stream-like objects, which is why I suggested the dynamic duck test that body.read is a function. Maybe extract this logic into a helper function so you can do appendToStream(response, body)?
| @@ -1,13 +1,15 @@ | |||
| var url = Npm.require("url"); | |||
| var crypto = Npm.require("crypto"); | |||
| var http = Npm.require("http"); | |||
| var toString = Npm.require("stream-to-string"); | |||
benjamn
Nov 20, 2017
Member
streamToString please
streamToString please
| @@ -1,13 +1,15 @@ | |||
| var url = Npm.require("url"); | |||
benjamn
Nov 20, 2017
Member
Let's just switch these tests to depending on ecmascript so we can use imports (or at least require) here.
Let's just switch these tests to depending on ecmascript so we can use imports (or at least require) here.
| clientDir: "/" | ||
| }; | ||
| Tinytest.addAsync("webapp - additional static javascript", function (test, onComplete) { | ||
| const run = async () => { |
benjamn
Nov 20, 2017
Member
I think you can just make the whole test function async, and call onComplete after the last await. I don't think Tinytest will do anything smart with the resulting Promise object, but that's fine.
I think you can just make the whole test function async, and call onComplete after the last await. I don't think Tinytest will do anything smart with the resulting Promise object, but that's fine.
| } | ||
| if (newHeaders) { | ||
| headers = {...headers, ...newHeaders }; | ||
| } |
benjamn
Nov 20, 2017
Member
Let's just use Object.assign(headers, newHeaders) here.
Let's just use Object.assign(headers, newHeaders) here.
| res.write(boilerplate); | ||
| res.end(); | ||
| }, error => { | ||
| stream.pipe(res) |
benjamn
Nov 20, 2017
Member
Does this need to be stream.pipe(res, { end: true })?
Does this need to be stream.pipe(res, { end: true })?
|
@benjamn thanks for the further notes! I'll try to get them done asap! |
|
@jbaxleyiii Thanks for all your work on this. Since you're busy now, I went ahead and addressed my latest feedback, and in the process I was inspired to improve |
PR #9343 changed the return type of Boilerplate#toHTML from String to Stream, which is likely to break existing code that expects a string. In order to make the change in return type more obvious, I have renamed the method to toHTMLStream, and I have attempted to update all call sites appropriately. However, because this change comes in the release candidate phase of Meteor 1.6.1 testing, it seemed important to preserve the string-returning behavior of toHTML, with a deprecation notice. Unless third-party code is using the Boilerplate class directly, I don't think the toHTML method will ever be called, and we can remove it in Meteor 1.6.2. Thanks to @macrozone for tracking this problem down. Fixes #9521.
|
I tried to replace I have Helmet, styled-components, and node-cache on the server side logic.
The problem still remains. Is there any other thing I need to do when switching over to PS: I see the server ran through the React Components (and if there were any warnings from propTypes, they also showed up in the server terminal console log when a browser try to access the site). |
|
It's been a minute but the removal of the boilerplate memoization is causing |
In initial testing, it results in a 40% reduction of TTFB🌮 (Time To First Byte [of taco]).
Todo
Wanted to get this up early enough for review though!
This PR address setting status codes including for SEO, improved alignment with react features, setting headers like eTag, and accessing headers and cookies