Roadmap To 1.0 And Beyond

[thescientist13]

This issue is intended to outline a the short and long term planning of the project for review and consensus gathering. It is broken down into the following milestones as part of the discovery done as part of the NoNo POC and the work to port that to Greenwood here in #417.

This will also serve as a way to update current GitHub Issues, Projects, Milestones

Questions

• <meta> component / package still needed? - yes! have some react-helmet like features
• RFC Declarative App templates still needed? - no, not really I don't think

Feature Parity (relatively) by 12/31/2020

This is work needed to accomplish the bulk of the work necessary to get #355 merged without much disruption to existing users. Merged into an ongoing release/1.0.0 release branch and tracked in the Architecture Digest Project / MVP Milestone.

Will want to validate these using an rc or next branch release before merging into master

• Greenwood Getting Started - chore/upgrade greenwood 0.10.0 greenwood-getting-started#22
• www.thegreenhouse.io - Enhancement/upgrade greenwood 0.10.0 thegreenhouseio/www.thegreenhouse.io#182
• projectevergreen.github.io - upgrade to 0.10.0 projectevergreen.github.io#36

Main thing here should be ensure that performance improves significantly and a number of existing issues in the backlog can be resolved. Each of the below numbered items would be a PR and tracked in the release branch PR.

1. PR and associated TODOs from Rfc/issue 355 no bundle development #417
2. Refactor serve.js into plugin based architecture bring done as part of [RFC] Plugins: Transforms #185
3. SPA support - delegated to Progressive Enhancement ("RedactJS") #354
   • (how to) handle strict / ssg optimization in greenwood.config.js
   • Decouple spa specific code from the bundle
   • Smaller / lighter router?
   • rollup dynamic input vars
   • puppeteer optional? e.g. SPA no SSG, SSG, vs SPA vs Hybrid (strict?)
4. GraphQL as a package, or something simpler
   • update all usages to use @data/client instead of fetch
   • data hydration still an issue
   • reduce duplicate graphql calls?
5. Handle loading prismjs - delegated to restore support for CSS bundling (w/Rollup) #427
6. How to handle user context options / default CLI templates/ - tracked in final release and questions checklist #428
7. Refactor serve.js into middleware based design - refactor prod server to use default standard plugins #519
8. create JSON and CSS plugins using import.meta.url - see [RFC] Plugins: Transforms #185
   • created issue for JSON plugin - have a plugin for being able to import JSON using ESM #516
   • created issue for image plugin - have a plugin for being able to import an image using ESM #517
9. Smarter handling of <title></title> tag, takes a function? - deferred to <meta> Component package #304
10. !Restore Polyfill and Google Plugins - will be deferred to [RFC] Plugins: Transforms #185
11. Babel / PostCSS / Browserslist support - created restore Babel / PostCSS / Browserslist for no bundle development #426
    • restore nested styles
    • enable for dev mode
12. Enable / get all specs passing - tracked in final release and questions checklist #428
13. Rollup refactoring - tracked final release and questions checklist #428
    • bundle everything from www/?
    • how to handle empty chunks warning from rollup?
    • avoid style.js file being generated by rollup CSS plugin
14. figure out why I can't use pages/blog/index.html fixed by Rfc/issue 355 no bundle development #417
15. confirm body[unresolved] is still an issue - see instances of <slot>-ed content and lit-element are rendering twice #433
16. confirm import-map support shim needed (for local dev) filter only JS packages for importMap and add tests #485
17. handling paths in templates, e.g. ./, ../, normalize everything to /?, also associated with route misses with using / - tracked in resolve (deeply) nested relative template paths to expected workspace path #435
18. Revisit all documentation tracked in revisit all technical documentation for no bundle development #430
19. Test more dependencies to ensure proper handling of CJS vs ESM based - tracked in validate / support resolution of various common packages in node modules #429
20. Publish to next branch and test in critical consumer projects - filter only JS packages for importMap and add tests #485

Roadmap to 1.0 by 12/31/2021

Things we should try and complete by end of year to achieve a 1.0. The aim is to complete low effort features that introduce / exercise new APIs, rounds out certain key features needed as part of the Feature Parity work.

1. Noticed that the CSS puppeteer (and whitespace from us?) puts into an index.html page is almost 10KB!!! This may be causing style collisions colliding with styles 👆 ? Either way, we should control the HTML output. If want to support inlining CSS great, but then we also shouldn't ship it over the network either. One or the other. Will make an issue to track this specific issue of junk in the trunk, and another to track optimizing for inlining CSS (may not work "work" for CSS-in-JS or else you get the FOUC thought...). - see Enhancement/issue 426 restore userland postcss #453 for more context
   • will leave this the way it is for now
2. Right now we have to process everything twice to support the user's PostCSS / Babel (once in serve and again in bundle / rollup since we read from file, not the server) in Rollup . It would be nice if Rollup could start from the processed output of the serve lifecycle so we don't have to re-run userland PostCSS again. In general, Rollup is doing a lot of inefficient work, would be great to refactor the whole thing from top to bottom. - see Enhancement/issue 426 restore userland postcss #453 for more context
   • not sure what exactly would was actionable here so not sure anything is needed here. I think by disabling pre-rendering as part of introduce Single Page Application (SPA) configuration mode  #518 , this duplication concern would be avoided.
3. better lockstepping between serve and rollup for knowing all pages, e.g. sharing graph.json, but still try and keep it async?
   • not sure what exactly would was actionable here, assuming the input for Rollup could just point to src/>. Nothing worth tracking for now, IMO.
4. Query Sorting / Filtering - do this server side and inject current page for grapqhl side / slimmer fetching? do this through the client instead (filter out the "dead pages" at the response level)?'
   • I think this was meant to be along the lines of a getServerSideProps type thing from NextJS. I think this is better captured as a 2.0 discussion topic instead of window.location.
5. unit testing development
6. Determine Node version and use of import, also (node:82640) ExperimentalWarning: The fs.promises API is experimental
   node_modules[(https://twitter.com/pikapkg/status/1306643472804278272).
7. https://www.reddit.com/r/node/comments/fidzj9/fdir_the_fastest_directory_crawler_for_nodejs_10k - not sure if the extra dep is worth it, tbh
8. replace JSDOM with something lighter / faster? - does it matter, only used for tests? - refactored a bit in Chore/issue 428 refactor smoke test and cases #497
9. Can be run via npx (0CJS - zero config JS) - confirmed as part of Website/issue 268 update project vision #499
10. Good First Issues for Hacktoberfest - N / A
    • serve task from the CLI
    • Add timestamp to guides
    • Html prettifier / formatter post optimize
    • Since Acorn 8.0.0, options.ecmaVersion is required. warning message
11. GitHub backlog
    • sourcemaps
    • Progressive Enhancement aka RedactJS - more like progressive refinement?
    • 404 Plugin
    • Capture intended API design - think post processors, HTML snippets, build lifecycles / orchestration
    • External Sources
    • website refresh
    • home page / project marketing
12. Validate if better HTML rendering is possible for second case - allow for markdown / HTML (or mdjs) authoring for blog posts thegreenhouseio/www.thegreenhouse.io#133
    • No special indentation
    • No restrictions on whitespace
    • Settings?
13. 1.0 Release Blog Post on project evergreen website
    • GitHub pages
    • do anything about CEA? - will add SPA support to Greenwood
    • https://www.youtube.com/watch?v=7yA7BGos2KQ
    • netlify analytics
14. Make sure all TODOs in code are resolved or tracked - done in Chore/issue 428 refactor smoke test and cases #497
15. eslint-plugin-lint - N / A
16. refactor develop vs prod server and to use standard plugins (if applicable) - refactor prod server to use default standard plugins #519
17. refactor liveServer into its own plugin. also, how to handle reloading for custom files e.g. .foo? - done as part of Server Plugin API #471
18. we need to add support for more font / icon types - extend format support for modern font and image extensions #521
19. should Greenwood standard plugins be pushed to plugins array? - consolidate all Greenwood standard plugin usage into one place #520
20. Handle inline <script> and <style> tags for develop (and prod, of course). Not sure if this can wait until post 1.0, but putting it on this roadmap for now. - inline <script> and <style> tags should get processed by Greenwood standard resource plugins #534
21. Favicons plugin, and add one to getting started repo - PWA Capabality #194
22. add support for @apollo/client in client.js - add full client API support for Apollo client in plugin-graphql (as ESM) #535
23. need to ensure a clear distinction between when file extensions are / aren't needed when using import as the spec says they should always be there but for now, will make it so node_modules can be bare, and can make an issue to add support for bare module specifier as a plugin - ensure a clear and consistent convention for when to use file extensions #536
24. should implement shared package.json dependency walker for importMaps like in plugin-babel / regenerator-runtime - create reusable walker, "package ranger" utility so that import map building logic can be shared #537
25. Currently need to have distinction between common and server client's until we can get to node v14 where we can use import on NodeJS code - Upgrade project codebase to ESM #532
26. Make plugin-graphql DRY for importMap and rollupAlias - refactor duplicate hardcoded dependencies list in plugin-graphql to a single shared data structure #538
27. Should really implement server.stop handling on process end - ensure proper termination of server plugins when Greenwood process ends #539
28. try and avoid need for createCache round-robin? 🐦 - optimize server side cache writing to reduce duplicate work and only run in production mode  #541
29. Upgrade Node v14 - Upgrade project codebase to ESM #532
30. eject functionality - done!
31. refactor plugin-babel to not have to require manual installation of Babel deps like regenerator-runtime
32. top level pages don't merge their own <title> / <meta> tags (bug) - top level pages don't correctly merge <title> tags #530
33. improve coverage coverage - get these branches up! and develop / serve tasks (+ server plugin / interface) - add tests for develop and serve commands #47
34. imports from markdown frontmatter - restore ability to import from frontmatter #529
35. default command should be build? (for npx) - discussion - Should Greenwood have a default command? #528
36. extract front matter contents from remark-frontmatter instead of separate frontmatter lib - get pages frontmatter from remark plugin instead of additional frontmatter package #531
37. rollup refactoring - hardcoded paths to .greenwood/, public/, path references, lot of (nested) loops - remove hardcoded directory references in rollup configuration #542
38. copy assets from code, or general copy plugin - create a general purpose copy mechanism like what is being used for assets/ directory copy #543
39. In the Getting Started repo (npm), seeing this warning from Rollup - upgrade rollup plugins and resolve preventAssignment default setting warning #526

     @rollup/plugin-replace: 'preventAssignment' currently defaults to false. It is recommended to set this option to `true`, as the next major version will default this option to `true`.

40. Should create an issue to track determining best optimization settings based on mode setting (at least by default). Not all scripts are created equally, preload vs prefetch. Router and GraphQL should probably use prefetch. - Default Optimization settings for Modes #545
41. Should make an issue to tracking adding per <link> / <script> overrides for inline and static optimization settings
    • and then use that to optimize the site further (all inline vs selective uses of inline + static). - enable support for all optimization settings and overrides #546
42. Should make an issue to track adding per <link> / <script> overrides for preload or prefetch hints - enable support for all optimization settings and overrides #546
43. For partial route filename hashing, might require a refactoring of how HTML optimization happens, since when doing it during serialization, you only assume that that one individual page has been built (since we are in a loop), and not the entire set, which means you can't use the graph to operate on the whole group to generate the same <routes for all pages. Saved a WIP branch here and will track as a follow up discussion. route partial filenames should be hashed based on file contents #547
44. Lighthouse test all pages - lighthouse audit all top level pages #524
45. Start a discussion about how to scale and apply a consistent design system over time to the website on pace for 1.0 release, to account for new features intended to be added that can be incorporated - Website redesign, branding, and long term structure #503
46. Would be good start a discussion on investigating opportunities to make pre-rendering less dependent on Puppeteer like creating a HydrateElement or looking into things like Declarative Shadow DOM - HydrateElement (SSR+Hydration for native `HTMLElement` / Web Components) #548
47. Browserslist: caniuse-lite is outdated. - browserslist is outdated #525

    Please run: npx browserslist@latest --update-db

48. On content change, need to refresh graphql server in development and generate new _graph.json) (Use mutations to patch on the fly???) - refresh and regenerate graph.json during development based on content structure changes #549
49. Make id unique to the route - Graph schema consistency enhancements #552
50. There is a double item.item in children query? - Graph schema consistency enhancements #552
51. Should only used Named exports - make all exports explicit by making them named exports #551
52. eliminate rollup plugin and use intercept? - Rollup plugin is a leaky abstraction #550
53. Starter kits (discussion) - Examples and Starter Kits #523

Post 1.0

These are tasks that can be worked on in the normal course of maintaining Greenwood and shouldn't require introducing any breaking changes. Really valuable features should have the Post 1.0 Milestone assigned so that contributors can get some visibility into where help could be applied. Post 1.0, we would probably moved to quarterly time boxed projects to track our status.

Maybe when we get to this point, we can all pick a couple tasks and start from there? We can keep this list flexible.
1, Image loading plugin

1. Explore strict mode for Greenwood website
2. generate import map as JSON file and import from a <script> tag / path, and only once - #import
3. Pre bundle dependencies for development (a la snowpack) - Pre bundling of node modules #628
4. Starter / Boilerplate / Generator (via npx), create a badge.io? - Examples and Starter Kits #523
5. From the backlog
   • open browser automatically
   • Performance automation
   • Differential Loading (IE11)
     • greenwood website home page loading has syntax error with internet explorer 11 (also causing no logo fade?) #198
     • website home page button is not colored blue in Microsoft Edge / IE11 (CSS variables support?) #196
   • 404 Plugin
   • Pre-processor plugins like SASS / SCSS. TypeScript support OOTB? esbuild? YML?
   • Image processing / lazy loading / general best practices (use of new URL and import.meta.url?)
   • CSS theming, constructable style sheets, inherited properties pierces the Shadow DOM
   • PWA
   • console / debug logging
   • HMR
   • video series
   • html minification / validation
   • Update meta on SPA route change
   • Have develop mode pick random port
   • Build Lifecycles
   • one off page templates as JS
   • linting
   • Reduce duplicate data fetches
6. better local HTTP caching w/ etag
7. Single File Components (SFC)
8. Standalone SPA / Dynamic Apps (https://youtu.be/RVNC61rkFxw) - enable a Single Page Applictation (spa) mode #589
9. Rollup code splitting by template
10. preloading / async / defer modules (optimizations) - Default Optimization settings for Modes #545
11. Inline JS / CSS
12. create a github action for puppeteer support
13. proxy dev server for API calls - create a reverse proxy server plugin #577
14. HTTP/2 for dev server - add http/2 support to development server #632
15. Restart graphql on file change? (Query, shelf) - hot reload Koa server for Greenwood development #633
16. Restart Koa server on file change (for local development) - hot reload Koa server for Greenwood development #633
17. Intelligent asset copy based on static analysis (e.g. favicon.ico)? Or just default to copying all of assets/ ?
18. use index.html as a default page template? - done!
19. Go all in on unified
20. ~ as a way to resolve node modules (as a plugin)?
21. dev server overlay (with build errors like what would be seen in the terminal) - Heads Up Display (HUD) for development #631
22. Canary pipelines by decoupling repo from website and having GitHub action build the website with latest version of Greenwood as part of each build (own org at this point?) - Canary Pipelines (Decouple website from monorepo?) #561

Thoughts for 2.0

Not necessarily breaking changes but also for more future facing ideas / features that might take a while or are just good for tracking at least? I think this should all be tracked in Trello for now until we have hit the 1.0 release at least. Then we can groom all of Trello and get it into a more refined GitHub backlog, which we should have by this time. Might also be worth creating an RFC specific repo for these (unless GitHub discussions becomes available by then).

• Web vital / lighthouse reports in Puppeteer as a debugging tool / plugin?
• SSR component (a la Nuxt / Next) - Server Side Rendering and Templating (puppeteer alternatives) #576
• Caching in between builds / Incremental builds - maybe using GitHub actions + a greenwood api to build a single route / page on demand?
• CSS Modules
• evaluate xm / posthtml / atomico
• web packaging?
• Streaming GraphQL data in / data out so server can start even if all data isn't ready yet as it becomes available / on demand
• Streaming / parallelized serialize and bundle phase, so we can rollup pages as they get serialized by puppeteer
• Would worker threads help our build process at all?
• Storybook support?
• serialize during development
• introduce a getServerSideProps type thing like from NextJS to help identify a page instead of window.location === route? - Server Side Rendering and Templating (puppeteer alternatives) #576 / Serverless (+ single route builds) #626
• decouple Rollup as the "initiator" for bundling so that Greenwood can just call out to any bundler instead of depending on Rollup to crawl all the HTML as the entry point.

Tech Radar

Just interesting things to watch, maybe in Trello or dedicated GitHub repo.

• Declarative SSR in Chrome, tweet from Jason Miller

[thescientist13]

I think we've got our mileage out of this ticket so going to close. A lot of good information has now been captured in our issues and discussions. v1.0.0 here we come!