Deploy remote rework #536
Closed
Deploy remote rework #536
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We are trying to make the kw follow some basic patterns. For this reason, this commit creates the deploy_main function and moves the parse function to the bottom of the file. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
This commit is preparation work to make the deploy feature slightly more generic by using better paths. Additionally, it moves those paths to a set of variables that can be easily updated, enabling us to give the user the option to set up the path they want in the config file. Closes: kworkflow#470. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Currently, kw expect that users conduct the following basic setup in the target machine: 1. Setup ssh connection to the root 2. Install required package (such as rsync) 3. Run any specific distro setup This commit introduces the `deploy --setup`, which will run all of the above basic configurations in the target machine. Now, the user only needs to guarantee the ssh via password the first time. Closes: kworkflow#481. Closes: kworkflow#220. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Codecov Report
@@ Coverage Diff @@
## unstable #536 +/- ##
============================================
- Coverage 73.84% 73.03% -0.82%
============================================
Files 32 34 +2
Lines 4687 5117 +430
============================================
+ Hits 3461 3737 +276
- Misses 1226 1380 +154
Continue to review full report at Codecov.
|
We want to make our deploy more generic, and one step for achieving this goal consists in adding bootloader detection. This commit introduces a function for handling the device and partition discovery which can be used to identify the disk bootloader and even provide essential info to users. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
This commit introduces a helper that identifies all hard disk types available in the system and saves them in a list. This can be useful for scanning each of the devices available in the system. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
If the target system uses MBR, there is a good chance that the first 4 bytes inform the partition table type. For this reason, this commit introduces a function that checks those bytes. It is important to highlight that we also added a couple of binary files to the sample folder for validating the new function and other news that will be implemented. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
This commit introduces a new function that tries to identify the bootloader information based on the MBR partition. Notice that this is heavily inspired in the bootscriptinfo. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Based on the standard set of files used by some bootloader this commit introduces a dedicated function for searching for these files and report the bootloader type. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
This function is responsible for collecting basic information from the target machine in order to provide data that kw can use to check if it can deploy the new kernel. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
kw deploy has to deal with generating rootfs and bootloader updates. Unfortunately, the logic that handles the kernel update is very tangled and mixes multiple steps spread around kernel plugin functions. This commit starts to untangle this code by decoupling: 1. Bootloader logic; 2. Specific bootloader file type (e.g., now we have grub.sh); 3. Move specific logic for generating rootfs to the distro file (e.g., arch.sh or debian.sh); 4. Isolate VM updates in their specific functions; Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
While conducting this massive code refactor, I took the strategy of first focusing on local and remote deploy even if it means breaking vm support. This commit represents a fix to the vm deploy, which means that kw vm is working again for Debian and Arch VM. Note that we still need to bring much more improvements to the VM parts; this commit is far away from being ideal for vm deploy. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
rodrigosiqueira
force-pushed
the
deploy-remote-rework
branch
from
482d5de
to
f28e230
Compare
The deploy test is one of the oldest files that we have inside kw, and it has a lot of technical debt. This commit is mostly focused on modernizing this test by cleanup and reorganizing it. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Our deploy code is not very well covered, and the tests have technical debt. This commit is mostly focused on testing new areas of the deploy feature and changing some other parts of the code. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
Plugins for kernel_install are outdated or do not cover important parts of the code; this commit reduces this gap by modernizing and adding more tests. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
rodrigosiqueira
force-pushed
the
deploy-remote-rework
branch
from
f28e230
to
a9adb18
Compare
When I tried to push the deploy refactor, I got the following error: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/p/pillow/python3-pil_7.0.0-4ubuntu0.4_amd64.deb 404 Not Found It looks like this is an update issue. Signed-off-by: Rodrigo Siqueira <rodrigosiqueiramelo@gmail.com>
|
This is a huge and important work refactor in the deploy feature, which still has a lot of things to be improved. I suspect that I'll not get any review on this PR any time soon; for this reason, I decided to merge it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
kw deploy is one of the most important parts of kw, and right now, this code is heavily coupled with distro-specific command and grub bootloader. This represents a barrier to making the deploy feature generic to other distros, systems, and bootloaders. This PR is a little bit big due to all the new code organization and other related changes. I still need to add more test coverage and documentation, but I want to open this PR in case that someone wants to comment.