Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamable Web Apps Presentation #54

Closed
23 of 38 tasks
justinbmeyer opened this issue Jul 14, 2017 · 18 comments
Closed
23 of 38 tasks

Streamable Web Apps Presentation #54

justinbmeyer opened this issue Jul 14, 2017 · 18 comments
Assignees

Comments

@justinbmeyer
Copy link

justinbmeyer commented Jul 14, 2017

Justin

  • - Finish Outline
    • - Introduction
    • - Stream + NDJSON + Fetch
    • - PUSH
    • - Incremental Render
    • - Conclusion

Jamie

  • - 1
    • show size perspective.
    • rid of question mark.
    • kayaker gradient
  • - 2 - rough draft
  • - 3-4 - add more rocks before, remove the rocks after
  • - 5-6 - animate the streaming part to show "chunking"
  • - 8 floating water
  • - 10 JUSTIN - take 5-6 animation and put it here
  • - 21 - Fix the line issue
  • - 8,21 - perspective
  • - 28 - make it look more like a waterfall ... rocks + rapids

Matthew

Stream + NDJSON + Fetch

  • - Which browsers support this technique? (Chrome, Safari)? (Bianca)
  • - What databases have good support for this: (Bianca)
  • - Performance Analysis (spreadsheet)
    • - 1st record and last record times across the following axes: (32 combinations)
      • With the technique and without the technique
      • Retrieving 50 and 500 records
      • Postgress returning every 50 and 500 record
      • Internet speeds:
        • Fast throughput vs slow throughput
        • Fast latency vs slow latency
    • - Capture video of best and worst performers (here)
  • - Library performance (Bianca)
    • - How does a react app perform at 100 and 500 records as each record is being added? Measure the cost of adding one record and the time it takes to update the DOM.
  • - Library support
    • - What frameworks / libraries can work with streams?
      • - Dash?
      • - Vue?

HTTP/2 PUSH

  • - Which browsers support this technique? (Chome, Safari, looks like edge)?
    • Chrome, Safari, Edge, and Firefox support HTTP2 and PUSH.
  • - Performance Analysis (spreadsheet)
    • - "JavaScript fully loaded" times across the following axes:
      • With push, without push, and without push but using a CDN
      • 50k of JS vs 500k of JS
      • A single JS asset, or 5 assets.
      • Internet speeds (it may not be possible to mix A and B with C).
        • A. Fast throughput vs slow throughput
        • B. Fast latency vs slow latency
        • C. A nearby consumer and a consumer far away
    • - Capture video of best and worst performers (here)
  • - Build something for StealJS to be easily able to push packages. (code examples)
    • - Get some example code in presentation.
  • - Library support .. do any build tools give you useful information?

Incremental Rendering

  • - Make it work with AJAX requests
@justinbmeyer
Copy link
Author

justinbmeyer commented Jul 14, 2017

@bgando / @matthewp I added a task around capturing videos for the best and worst performers (biggest deltas). It will probably be useful if I can show the comparisons side by-side, so make the videos about 2x as tall as they are wide.

For example, I could show:

  • NDJSON Streaming 500 records of a 250k data set, on a 3g connection

VS

  • SINGLE RESPONSE of 500 records of a 250k data set, on a 3g connection

Also, the app might look as simple as a bunch of very short rows, each with their id. Similar to:

streamable_web_apps_v2-justin

But where you might see ~60 of the 500 rows instead of ~7

@justinbmeyer
Copy link
Author

justinbmeyer commented Jul 15, 2017

@matthewp I'm thinking instead of react, maybe lets just get a "basic" app working like:

module.exports = {
        // what is `createState` for?
	createState: function(request){
		return {};
	},
	render: function(document, state) {
		document.innerHTML = `
		    <div id='sidebar'>Pet Store <div id='cartCount'></div></div>
		    <div id='products'></div>`;

		async function listProducts(){
			var response = await fetch('/products.ndjson');
			var productsEl = document.getElementById('products');
			for await (const product of ndjsonStream(response.body) ) {
				var productEl = `<div>{product.name}</div>`;
				productsEl.appendChild(productEl)
			}
		}
		listProducts();

		fetch('/cartCount').then(function(response){
			return response.json();
		}).then(function(data){
			document.getElementById('cartCount').innerHTML = data.value;
		});
	}
};

I'd like to show how to setup something like this with done-ssr. I 'think' we made this work with the existing SSR. Can you give some sample code of how something like this could be setup to work? Thanks!

@matthewp
Copy link

matthewp commented Jul 17, 2017

I can create an example app project like this, I agree that it's a good way to go. Would look like:

var ndjsonStream = require('can-ndjson-stream');

module.exports = function(request){
	document.body.innerHTML = `
		<div id='sidebar'>Pet Store <div id='cart-count'></div></div>
		<div id='products'></div>`;

	async function listProducts(){
		var response = await fetch('/api/product');
		var productsEl = document.getElementById('products');
		for await (const product of ndjsonStream(response.body) ) {
			var productEl = `<div>${product.name}</div>`;
			productsEl.appendChild(productEl);
		}
	}
	listProducts();

	fetch('/api/cart').then(function(response){
		return response.json();
	}).then(function(data){
		document.getElementById('cart-count').innerHTML = data.value;
	});
};

@justinbmeyer
Copy link
Author

justinbmeyer commented Jul 17, 2017 via email

@matthewp
Copy link

matthewp commented Jul 18, 2017

Chrome devtools lets you adjust latency and throughput independently.

screen shot 2017-07-18 at 12 58 46 pm

@matthewp
Copy link

I've collected the data on HTTP2 performance. You can view the spreadsheet here.

High level observations:

  • CDN does better than PUSH in most situations but:
    • CDN has the biggest advantage when both latency and throughput are poor.
    • PUSH does better when throughput and latency are good.
    • PUSH does better when latency is good and the user is far away.
    • CDN Does better when latency is poor and user is far away.

@matthewp
Copy link

I've uploaded 3 videos which show some faster and slower examples from the data.

The datacenter for these tests is in San Francisco.

Fast - 5 Files, 50k gzipped total, 750k throughput 2ms latency, from Singapore, PUSH

link

Fast - 1 File, 50k gzipped total, 30mb throughput 2ms latency, from Kentucky, CDN

link

Slow - 5 files, 750k throughput 300k latency, from Singapore, No PUSH

link

@matthewp
Copy link

matthewp commented Jul 24, 2017

NDJSON data set is here: https://docs.google.com/spreadsheets/d/1_uO79zTYfPwfv3wcOp8xuYTx19KBdw7xp-4RplWnb34/edit#gid=0

Biggest things I noticed:

  • NDJSON is a huge win when there is a large dataset and poor throughput.
  • NDJSON is a win when latency is low.

@matthewp
Copy link

NDJSON Videos

These videos show the difference between using NDJSON vs JSON at 500 records, streaming 1 at a time with 750k throughput and 2ms latency.

@justinbmeyer
Copy link
Author

justinbmeyer commented Jul 24, 2017 via email

@matthewp
Copy link

i'm going to try.

@matthewp
Copy link

screen shot 2017-07-25 at 9 51 11 am

@matthewp
Copy link

I'm recalculating the numbers using this sql query to slow it down:

var COUNT = 500;

var cc = `
	SELECT t.*
	FROM (
		SELECT *, row_number() OVER(ORDER BY id ASC) AS row
		FROM todos
	) t
	WHERE t.row % 500 = 0
	limit ${COUNT}
`;

@matthewp
Copy link

Complex query

I created two new videos using the above query. The spreadsheet is here.

The times are below:

NDJSON - First Record NDJSON - Last Record JSON
140.57 509.05 535.36

Videos

@bgando
Copy link

bgando commented Jul 26, 2017

image
Sauce Labs

@matthewp
Copy link

I'm going to make a branch of donejs-streaming-product-page that uses basic HTML and provide a sample as asked for here.

@matthewp
Copy link

This is a code sample a project using basic HTML.

@bgando
Copy link

bgando commented Jul 26, 2017

Vue.js code sample

@bgando bgando closed this as completed Aug 31, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants