[Moved https://github.com/netlify/build/issues/39] Improve heuristics detection

[DavidWells]

In an effort to speed up build times, we need to trim down the build image used as much as possible.

Having advanced heuristics of what the projects needs to run should help.

There are a couple of different methods we can use for this:

• Parsing & resolving build commands
• Leveraging https://github.com/netlify/netlify-dev-plugin/tree/master/src/detectors
• Allowing users to specify runtimes needed as env variables.
• Allowing users to specify custom docker images

We could also run the first build with the kitchen-sink image and use ML to optimize the image on the next run

[swyxio]

we need to split out netlify dev detectors asap. cc @RaeesBhatti i can probably do it but just fyi

[DavidWells]

The new built repo will likely be a mono repo with individual packages

Heuristics lib will probably live in there

[swyxio]

could we brainstorm the output needed from the heuristics?

• language
• package manager
• likely run commands
• static dir
• publish dir
• devserver port

to get even better customizability, we might need to pass in options to the heuristics that are then run by the project, which then impact the output above. this flow is best owned by the heuristic lib.

eg, i want to run gatsby on port 6969, i pass it in to to heuristics lib, it actually runs it with the right flag e.g. --port 6969, and then outputs devserver port as 6969. very simplistic here but probably the most common usecase.

[DavidWells]

Yeah for sure.

The heuristics are used in two (many more in future) places:

• Starting the dev env netlify dev
• At the beginning of a build netlify build

It would parse the project and try and make it's best guess on what the src code, ENV vars, and netlify config tells it.

So all the existing fields we have but with some additions like:

• project version (ie gatbsy v1, v2.3.22 etc)
• local bin path (locally available binaries to run)
• ...

Not sure on all the fields yet. Open to whatever

The other changes to the lib will likely be:

• Optimizing fs calls (making everything async)
• Passing in relevant data into detectors as arg (pkgJson, requirement.txt file contents, etc)
• Passing in /utils/jsdetect as function args for external/third party detectors to leverage

[ehmicky]

Related: #39

[verythorough]

Closing in favor of #39, which is also now closed.