Roadmap 2021

[michaelrambeau]

A discussion about some directions to improve Best of JS in 2021.

Short-term features

Let users compare projects

Now it's difficult to compare data between projects, besides data visible from the search results, displayed inside the table.

It could be useful to compare some related projects showing graphs about:

• the evolution of the stars on GitHub (weekly, monthly, yearly)
• the variations of the number of downloads on NPM
• the size of the bundle
• ...

The idea would be to be able to open a page that combines data about several projects.

It could be a page /compare?react+vuejs+angular to compare the 3 leaders of the UI framework category for example.

The most difficult question is how to access these pages? We could "suggest" some related or similar projects from a given project's page.

We could also add a kind of "project picker" component to let the user pick the project she wants to add to the mix.

Handle an history of projects visited by the user?

I often check the same projects again and again, but since it's only possible to display one project at a time, it's painful to go back to a previous project, the only solution is to search for it... again!

The simplest solution would be to show somehow an history of the projects opened by the user, that would be persisted in the user's browser.

Long term: a new architecture?

The architecture of the application limits the number of projects we handle because the list of all available projects is handled by a single huge JSON file: projects.json (it contains about 1500 projects, its size is about 150 kB gzipped).

This JSON file provides a context shared by all pages of the application.

Should we move toward a more "server oriented" approach? (or "less Single-Page-Application approach")

I'm thinking about Next.js because we could take advantage of its SSR features to improve SEO.

The most difficult feature to re-write is the search feature: the current search feature is very fast because there is no client - server round trip, everything happens in the browser. Basically the current search is a simple Array.filter we perform against the projects.json file mentioned above.

One possibility would be to use Algolia like all the cool kids do, instead of implementing our own search.

[L1lith]

Another solution would be to have a server that would automatically generate the JSON for each project individually, also it could statically cache the content of the front page. It would upload these static files to a hosting service automatically. Also I don't think most things should be cached for too long as packages and their statistics are subject to change.

[davidjerleke]

Hi @michaelrambeau!

Have you considered replacing the current search implementation with Algolia search?

I’ve tried it and I think it works really well. The open source license is free but you have to apply for it.

Cheers,
David

[michaelrambeau]

Hello David @davidcetinkaya it's good to see you again! 😄
Yes I mentioned that possibility at the end of the initial comment, thank you for the details, it's good to know.

An other possibility is to follow the approach used by MDN for their search feature: How MDN’s autocomplete search works.

It seems they use a big JSON flle only for the search feature. So instead of having all data about projects in the JSON file like we have now, it could be limited to data "indexed" when users enter text in the search box (project's title, package name and keywords in the description maybe). That file could be lazy-loaded when users start to search for something.

What Mozilla did is interesting, I've just checked in the browser: when the mouse is above the search box, the search-index.json JSON file is loaded by XHR.

https://developer.mozilla.org/en-US/search-index.json

So I will consider those 2 options when changing everything! Now I'm currently working on adding a dark mode feature, I got something working but the size of the bundle worries me, see the details here: #112

[davidjerleke]

Hi @michaelrambeau!

Haha sorry about that. You did mention Algolia so my bad. In that case, let me just leave it with that I think Algolia works quite well. But on the other hand, I haven't tried the Mozilla approach 🙂.

Cheers,
David

[jsun969]

Hi @michaelrambeau . I love bestofjs and often browse it in my spare time! I am a fullstack dev myself. Do you have some plan to refactor the backend (I saw it was made of js) to ts? Looking forward to contributing to this project!
btw, how do you storage such huge data? xD

cc @nvh95

[nvh95]

@jsun969
Thank you for your interest in BestOfJS. Please join our Discord server at https://discord.com/invite/rdctdFX2qR, we can discuss more details here.
Is the backend you mentioned https://github.com/bestofjs/bestofjs-backend? In my opinion, I think it's great if we can convert it to Typescript. What do you think @michaelrambeau

@michaelrambeau
@jsun969 is a young talented engineer who is 17 years old, currently based in China. He built React Hook Form Devtools Chrome Extension (https://github.com/react-hook-form/devtools-extension) and his own React modal management system: React Hook Dialog (https://github.com/jsun969/react-hook-dialog). Please feel free to welcome him to our project.

[michaelrambeau]

Hello Justin @jsun969 thank you for your kind words!

Thank you for your interest in the project, it would be great to improve the backend, making it more robust and standard, adding TypeScript like @nvh95 suggested (thank you Hung for the introduction!)

First I need to spend some time to add more context, describing how it works, let's discuss on Discord!

[michaelrambeau]

Closing the issue as we have managed to reach some of the goals, Best of JS now runs on top of Next.js (see #176 )