Combine package.json and target/frontent/package.json

[caalador]

We should make package.json fully generated by flow and move the information in target/frontend/package.json into package.json so that the plugin handles all dependencies in one place.

This would mean that any user modification directly on the package.json file might be lost on an update by flow, but it would make the developers life easier as then the knowing which modules the project is actually depending on is easier and keeping track on versions can simply be done in version control.

Also this will simplify our code as we do not need to do hashes for dependencies to know if we need to run npm i after a mvn clean

Part of #6966

[Artur-]

This would mean that any user modification directly on the package.json file might be lost on an update by flow

In which case would something be lost? Shouldn't Flow know which parts it "owns" and which parts is owned by the user?

[caalador]

Dependencies are removed if they are not in the found dependencies map (e.g. all user changes).
It was proposed to have a flag for cleaning extra dependencies automatically by default and having the possibility of having the framework only add dependencies which would make the package.json user editable for all parts except for any annotated packages. (that's basically true also now as version collisions will probably break the build)

In the case where the user doesn't want auto removal they would be responsible for fixing any typos as highlighted by Manolo the comment in #6966

[Legioth]

Would this mean that if read some guide online and blindly follow the instructions to run npm install --save left-pad, then my dependency will be removed the next time I start the server?

[caalador]

By default it would, as it wouldn't be found on the java side. Or then the default should be only add, but then any typos in annotations would not be fixed in the package.json making the user need to fix the dependencies.

[Legioth]

Let me propose a slightly more complicated design that would work in all cases that I'm aware of.

Introduce a vaadinDependencies field in package.json that contains a copy of anything that Vaadin as added as a regular dependencies. Based on this, Vaadin is only allowed to remove (or update) values in dependencies if they still match the entry in vaadinDependencies. The structure of vaadinDependencies can either be exactly the same JSON structure as for regular dependencies, or alternatively encoding that JSON as base64 to discourage users from eidting it manually.

In this way, something that you add manually using e.g. npm install --save left-pad will never be removed by Vaadin. At the same time, adding @NpmPackage(value="left-bad", version="1.3.0") would cause a corresponding entry to be added also to vaadinDependencies which means that once the typo is fixed, Vaadin would see that there's a Vaadin-managed left-bad dependency but no corresponding @NpmPackage annotation. Based on this, the bad dependency could be safely removed.

If the user has manually updated a version number for a dependency that was originally added by Vaadin, then the entry in vaadinDependencies would not match and the dependency would thus not be removed automatically. If the user then updates the version number in @NpmPackage to a version newer than the one manually defined in package.json, then Vaadin would log a warning about a potential version mismatch during startup.

[caalador]

This would be acceptable overhead, and still a lot simpler than the current setup.
To also satisfy #6907 at the same time would be to add the full object

"vaadin": {
  "dependencies": {},
  "devDependencies": {}
}

Then we could accept manual changes to devDependencies also with minimal changes in functionality.

[caalador]

Acceptance criteria:

• target/frontend/package.json is removed and all dependencies are added to package.json
• package.json gets a new JsonObject vaadin that contains dependencies and devDependencies that are the framework managed packages.
• managed packages in vaadin should accept newer versions by users to dependencies, but override too old versions
• Framework should not touch dependencies that are not found in vaadin dependencies.