Improve 4 second load time

[LudwikJaniuk]

We get this when we run npm run dev:

Warning: data for page "/[dataset]/[dataView]" (path "/utslappen/karta") is 2.65 MB which exceeds the threshold of 128 kB, this amount of data can reduce performance.

I am guessing this has to do with the map data. More info is here: https://nextjs.org/docs/messages/large-page-data

But is it so bad? I ran a speedtest which does point out we are suffering from ~4 second load times. This can make someone lose interest if they are on the phone.

So it would be useful to investigate if there are things we could change to improve the load time. The "large page data" issue next.js points out is a good place to start, but we need to measure before/after to verify if that makes a difference, or if we need to look at other stuff as well.

[Greenheart]

Ideally we should only load the required dataset for the requested route, instead of responding with all datasets. This is the obvious first step that would greatly improve performance.

For example, if /koldioxidbudgetarna/karta is requested, we should only respond with the required data to render koldioxidbudgetarna.

Other optimizations that could reduce the payload size further (with more work required):

The returned JSON data has many long key names that are used in almost every object. If we reduced the key length, we reduce the amount of data transferred by approximately 10%

We could also consider if we want to have as many decimal points for numbers. For electricCarChangeYearly for example, we send up to 18 decimal points, but only render 1 decimal point in the UI. Why not return the rounded data directly from the server? This could likely save quite a bit of data.

This also goes for all floating point numbers, for emission reductions and more. If exact sorting is important, we could do that on the server side when preparing the dataset. Or even better, pre-process the data in Python so we have both a raw dataset and an optimized version for usage on the website.

To solve potential inconsistencies, we will always be able to provide a raw dataset if anyone asks for it. But the website only cares about the actual data to display. Better to make the raw data available via a JSON download, separately from the Next.js page data.

[LudwikJaniuk]

Good ideas @Greenheart.

[mathiasb]

Could be possible to create a more consumtion friendly version of the file with a small Transformation script standalone to begin with.

[N-Asadnajafi]

I picked up this task @LudwikJaniuk . It seems there is still room for improvement. As a good practice in Front, I will take advantage of Redux and will separate the logic from the rest of the app. By minimizing unnecessary re-renders, the performance will improve significantly. And as a result we have faster load times and smoother interactions. Overall we will have a better user experience.

[LudwikJaniuk]

Sounds good and good luck! Please communicate with @Greenheart as you go along to ensure what you're working on is in line with other ongoing work.