Improve loading times

[AlexxNica]

Overview

Users currently face slow loading times on the homepage when submitting a new URL to generate a roadmap. We should explore the root cause and fix this.

Possible improvements

• Decouple functional components from the API and reuse them on the client without having to query the API
  • https://nextjs.org/docs/basic-features/data-fetching/get-server-side-props#getserversideprops-or-api-routes
• Move from server-side rendering to client-side data fetching
  • https://nextjs.org/docs/basic-features/data-fetching/client-side

Open question

• When do we call this done?

Related discussions and proposals

• feat: Async loading of issues #103
• Decouple functional components from the API and reuse them on the client without having to query the API. #180
• feat: make api/roadmap call from client #181
• feat: load children asynchronously #182
• feat: Load-time under 1 sec #271
• Data fetching on client-side directly from GitHub API #325

[SgtPooki]

related to #103, but #103 is more of a sub-item for this.

[SgtPooki]

ended up having to resolve most of this as part of #158 , and will be targeting resolving this with #182

[AlexxNica]

Reopening this issue as the suggestions still need to be addressed. The improvements are related to making calls directly from the client to GitHub without going through our API.

I've opened #325 to test this approach, and I think we can use client-side fetching to load data from our APIs after the client hits GitHub's rate limiting if needed.

[AlexxNica]

Related discussions and proposals:

• feat: Load-time under 1 sec #271
• Data fetching on client-side directly from GitHub API #325

[AlexxNica]

I'm having some ideas on strategies we can use to improve loading while I work on #325:

1. We load the first roadmap page (simple view) on the client side using the standard limits from non-authenticated GitHub API calls. (It's likely that most simple-view roadmaps would be within the limit range.)
2. After loading the first view, we start prefetching and preloading data for the roadmap's children on the background. As soon as the user clicks on the detailed view, we'd already have most if not all of the data loaded.

For the background work, we can fetch data from GitHub using a mix of methods (e.g., authenticated & non-authenticated calls to the REST and GraphQL endpoints). We defer calls as much as possible to save resources.

Currently, I think the main features that require data from a roadmap's children are:

• For the overview/simple view: the ability to see the completion rate and click to open the item as a roadmap if the item has children.
• For the detailed view: the ability to see the root roadmap's children as themes with grandchildren as subitems under those themes.

Given that info, we can think of different approaches to this.

• One idea would be to gate the detailed view for users authenticated with a GitHub account.

Note: I'm just thinking out loud here, as we need more data to make an informed decision.