Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finalize assignments: Chapter 18. Page weight #20

Closed
3 tasks done
rviscomi opened this issue May 21, 2019 · 9 comments
Closed
3 tasks done

Finalize assignments: Chapter 18. Page weight #20

rviscomi opened this issue May 21, 2019 · 9 comments

Comments

@rviscomi
Copy link
Member

rviscomi commented May 21, 2019

Section Chapter Author Reviewers
IV. Content Distribution 18. Page weight @khempenius @henrisgh @tammyeverts @paulcalvano @flowlabs

Due date: To help us stay on schedule, please complete the action items in this issue by June 3.

To do:

  • Assign subject matter expert (author)
  • Finalize peer reviewers
  • Finalize metrics

Current list of metrics:

  • Distribution of resource size: (p10, p25, p50, p75, p90) x (total, JS, CSS HTML, Fonts). In addition, how this has changed over the past year.
  • Distribution of resource quantity: (p10, p25, p50, p75, p90) x (total, JS, CSS HTML, Fonts). In addition, how this has changed over the past year and since the release of H2.
  • Very Optional: H2's impact on resource quantity.
    • Determine which sites serve the majority of their first-party content using H2. (This % alone would be interesting. I also wonder if there would be a significant difference between using .5 and .9 as the threshold.)
    • For those sites using H2, look at how CSS & JSS resource quantities varied before and after H2 adoption.

👉 AI (@khempenius): Finalize which metrics you might like to include in an annual "state of page weights" report powered by HTTP Archive. Community contributors have initially sketched out a few ideas to get the ball rolling, but it's up to you, the subject matter experts, to know exactly which metrics we should be looking at. You can use the brainstorming doc to explore ideas.

👉Optional AI (@khempenius): Peer reviewers are trusted experts who can support you when brainstorming metrics, interpreting results, and writing the report. Ideally this chapter will have multiple reviewers who can promote a diversity of perspectives. You currently have 1 peer reviewer.

The metrics should paint a holistic, data-driven picture of the page weight landscape. The HTTP Archive does have its limitations and blind spots, so if there are metrics out of scope it's still good to identify them now during the brainstorming phase. We can make a note of them in the final report so readers understand why they're not discussed and the HTTP Archive team can make an effort to improve our telemetry for next year's Almanac.

Next steps: Over the next couple of months analysts will write the queries and generate the results, then hand everything off to you to write up your interpretation of the data.

Additional resources:

@rviscomi rviscomi transferred this issue from HTTPArchive/httparchive.org May 21, 2019
@rviscomi rviscomi added this to the Chapter planning complete milestone May 21, 2019
@rviscomi rviscomi added this to TODO in Web Almanac 2019 via automation May 21, 2019
@rviscomi rviscomi changed the title [Web Almanac] Finalize assignments: Chapter 18. Page weight Finalize assignments: Chapter 18. Page weight May 21, 2019
@rviscomi rviscomi moved this from TODO to In Progress in Web Almanac 2019 May 21, 2019
@rviscomi rviscomi assigned khempenius and unassigned paulcalvano May 21, 2019
@rviscomi
Copy link
Member Author

Added @henrisgh (Henri Helvetica) as an author 🎉 and invited to @HTTPArchive/authors

@rviscomi
Copy link
Member Author

Added @flowlabs as a reviewer 🎉

@rviscomi
Copy link
Member Author

rviscomi commented Jun 4, 2019

@khempenius @henrisgh @paulcalvano @flowlabs we're hoping to have the list of metrics for each chapter finalized today. There are only 3 metrics listed for this chapter at #20 (comment). Could you take one last look and when you think it's in a good place tick the last TODO checkbox and close this issue? Thanks!

Also @henrisgh could you please go to https://github.com/HTTPArchive and accept your invitation to the Authors team? This ensures that you get author-specific communications, I can assign issues to you, and you can edit issues like this one.

@khempenius
Copy link
Contributor

TLDR; I'll take care of updating & closing this issue. For those curious below was my thought process on this:

I thought the entire analysis and writeup was due today so these were the metrics I ended up already calculating:

  • Distribution of resource size: (p10, p25, p50, p75, p90) x (total, JS, CSS HTML, Fonts). In addition, how this has changed over the past year.
  • Distribution of resource quantity: (p10, p25, p50, p75, p90) x (total, JS, CSS HTML, Fonts). In addition, how this has changed over the past year and since the release of H2.

I found that the number of CSS & JS resources has increased moderately since H2's release, but I don't think that's enough to draw a conclusion from (It is it H2? Or is it just sites getting larger?)

If we wanted to go deeper on this H2 detail (this is interesting, but I don't think it's essential to talking about page weight), I would propose these metric(s):

  • Determine which sites serve the majority of their first-party content using H2. (This % alone would be interesting. I also wonder if there would be a significant difference between using .5 and .9 as the threshold.)
  • For those sites using H2, look at how CSS & JSS resource quantities varied before and after H2 adoption

Web Almanac 2019 automation moved this from In Progress to Done Jun 4, 2019
@rviscomi
Copy link
Member Author

rviscomi commented Jun 4, 2019

No just the metrics are due today 😅 See this timeline for a better idea of when everything comes together.

Thanks for closing this out Katie!

For those sites using H2, look at how CSS & JSS resource quantities varied before and after H2 adoption

We do have historical data but no guarantee that the pre-H2 data exists for a given page. The URL corpus thrashed quite a bit over the last year as we transitioned from Alexa to CrUX. So these results might be inconclusive.

@rviscomi
Copy link
Member Author

rviscomi commented Jun 4, 2019

Two other things came to mind:

If you're interested in writing the queries for this chapter, can I also add you to the Data Analyst team? Would you also be interested in writing the queries for #21?

In case you have an early draft ready to go, that might actually be helpful as a placeholder for the UX teams to start exploring data viz and other design options. Would you be open to sharing what you have so far?

@khempenius
Copy link
Contributor

khempenius commented Jun 4, 2019 via email

@rviscomi
Copy link
Member Author

rviscomi commented Jun 4, 2019

Great thanks! Added you as an editor to the Metrics Triage sheet and put your name down for this chapter's metrics. More info about triaging in #33.

@rviscomi
Copy link
Member Author

Good news, @tammyeverts is signing on as a coauthor! 🎉 Thanks Tammy! I've also sent you an invite to the Authors team on GitHub. You can accept by visiting https://github.com/HTTPArchive/

(@henrisgh you've also got an outstanding invitation to the Authors team, so follow the link above to accept.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

No branches or pull requests

4 participants