Proposal for simplifying repetitive tasks of the development, test, lint, format, build and flash processes

[qlrd]

When reading the official documentation over and over again (initially, with the intention of translation), I noticed that some tasks (development, build, lint, update, test, etc) require, on an extended time scale, a considerable amount of repetition of the commands.

I think this could be simplified, through a Makefile.

I didn't open a PR to not affect some improvements being made by @jreesun and @odudex , but I share a gist that, apparently, simplified the tasks.

Some of the simplified tasks:

• make update: replacing git pull origin main && git submodule update --init --recursive
• make install: replacing poetry install and some optional dependencies (in this sense, a proposal to modify pyproject.toml to)
• make test: replacing poetry run pytest --cache-clear --cov src/krux --cov-report html ./tests
• make test/verbose: replacing poetry run pytest --cache-clear --cov src/krux --cov-report html --show-capture all --capture tee-sys -r A ./tests
• make build/sign: replace some tasks to installing from the source:
  • vagrant up
  • vagrant ssh -c 'cd /vagrant; ./krux generate-keypair'
  • vagrant ssh -c 'cd /vagrant; ./krux pem-to-pubkey pubkey.pem'
  • copy the hex pubkey
  • replace in src/krux/metada.py file
• make build/maixpy_m5stickV: replace vagrant ssh -c 'cd /vagrant; ./krux build maixpy_m5stickv'
• make build/maixpy_amigo_ips: replace vagrant ssh -c 'cd /vagrant; ./krux build maixpy_amigo_ips'
• make simulator: run and logs simulation process

To see all options, type make help

I also prupose a change on simulator/run-all-sequences.sh, with the characteristic of reducing the console output and allowing the writing of log files during simulation.

[ghost]

Great idea! I think that would be extremely useful and help to organize all the commands.

One important issue, however: I want people on Windows, Mac, and Linux to all be able to pull down the code, build from source, and flash to their devices. That is a large reason for why the project uses Vagrant in the first place. As it is now, if you're on a vanilla Windows platform, all you need to do is download vagrant and git and you'll be good to go. With the introduction of a Makefile, this could complicate the process for Windows users.

Is there a way to make this easy to run on Windows without requiring a lot of additional effort? Ideally, supporting it would require at most one more .exe to install.

[qlrd]

One important issue, however: I want people on Windows, Mac, and Linux to all be able to pull down the code, build from source, and flash to their devices.

A very important issue. In theory, Mac users can run the make commands smoothly since OSX is a Unix Like. At most they will need to install something through brew. But i'm not a mac user...

Is there a way to make this easy to run on Windows without requiring a lot of additional effort? Ideally, supporting it would require at most one more .exe to install

I believe there are close possibilities:

• Replace a Makefile to:
  • a Rakefile (taking advantage that vagrant runs in ruby environment, and it is possible to detect OS );
  • a similar python tool, like pypyr
• Usage of windows chocolatey (see here);
• Usage of nmake with a custom Makefile.win (see here)
• Usage of cygwin(see here)
• Usage of NSIS (see here)

EDIT:
An installer build here

[odudex]

Isn't it possible to just login Vagrant, than run all the commands as you were on Linux?

[qlrd]

Taking the first short steps. I think I found a node.js framework -- electron-builder -- that makes it possible to create programs for Mac, Linux and Windows using a separate instance of chrome.

I think now it's integrating the build process (vagrant, etc.) and installation steps in .js scripts

Isn't it possible to just login Vagrant, than run all the commands as you were on Linux?

Yes, its possible. But I'm kind of lazy person (:face_with_head_bandage: ) and I like to type few commands when they are repetitive.

[ghost]

The krux shell script was written so that it could be run either on a linux machine directly or inside Vagrant.

Because I was trying to avoid platform-specific instructions (and troubleshooting), the website documentation for installing from source just says to run the krux script inside vagrant with the vagrant ssh -c 'cd /vagrant; ./krux prefix before all commands.

As for the development of Krux, it's less important that it be cross-platform. So, the Makefile could be written assuming it will be run on linux or inside Vagrant just like the krux script is, and the documentation for installing from source on the website could remain as it is. But the development instructions on github could be updated to refer to the new Makefile commands.

Would that be easiest? That may have been what @odudex was suggesting.

[ghost]

A GUI installer based on Electron would be pretty neat if it could automate the installation commands.

Something like this:

• a Makefile to make development easier and faster
• a cross-platform electron GUI installer to make installation easier and faster (for less technical users)

Is that what you're proposing?

[qlrd]

As for the development of Krux, it's less important that it be cross-platform. So, the Makefile could be written assuming it will be run on linux or inside Vagrant just like the krux script is, and the documentation for installing from source on the website could remain as it is. But the development instructions on github could be updated to refer to the new Makefile commands.

Would that be easiest? That may have been what @odudex was suggesting.

I think it will be.

Something like this:

• a Makefile to make development easier and faster
• a cross-platform electron GUI installer to make installation easier and faster (for less technical users)

Is that what you're proposing?

Sorry for not expressing myself properly before. But yes, that's exactly what I wanted to propose. In this sense, a GUI could (in a ideal and a similar way to what Raspiberry Pi Imager does):

• Give the option to write pre-built firmwares to a microSD;
• Give the option to flash the firmware on a supported device;
• Give the option to install from source code. In that case, the Electron app would also do:
  • Download and extraction of .tar.gz or .zip;
  • sign, build (and other) processes described in the documentation, via the child_process.spawn method
    • In case of windows, run vagrant through child processes

[ghost]

That sounds great to me! To keep things organized, can you create two issues: one for the Makefile with a small explanation about it (similar to your original post here), and one for the installer? That way we can track their development separately (ideally in two separate PRs)

Going to convert this issue into a discussion