Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge rhel9 branch into master #178

Merged
merged 115 commits into from Nov 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
6c6c289
Bump version after release.
matejak Jun 1, 2020
df5610b
Merge pull request #116 from matejak/bump
ggbecker Oct 6, 2020
1de759e
Expanded group data to detect more package collisions.
matejak Nov 4, 2020
0403d4f
Merge pull request #128 from matejak/improved_package_groups
ggbecker Nov 9, 2020
35d4a07
Fixed issues with creating translations patches.
matejak Mar 2, 2021
43ce5ea
Use the password policies from DBus
poncovka Jan 20, 2021
c544194
Merge pull request #137 from poncovka/master-replace_pwpolicy
matejak Apr 23, 2021
2d1be79
Add packages
jan-cerny May 21, 2021
3704c1a
Merge pull request #138 from matejak/translations_fix
jan-cerny May 26, 2021
eb0ceef
Merge pull request #143 from jan-cerny/fedora_testing
matejak May 26, 2021
43b5365
Document reanaconda
jan-cerny May 24, 2021
d732b75
Merge pull request #144 from jan-cerny/reanaconda_doc
matejak Jun 4, 2021
b1b24ff
Replace DataStreamHandler and BenchmarkHandler
jan-cerny Jun 3, 2021
e445f59
Raise error when file not on the server.
matejak May 26, 2021
b7d8fb1
Moved wait_and_fetch_net_data from common.
matejak Jun 4, 2021
c0b9302
Add comment
jan-cerny Jun 7, 2021
38296b6
Merge pull request #145 from matejak/fetching_fix
jan-cerny Jun 7, 2021
e24b305
Change way of processing tailoring
jan-cerny Jun 7, 2021
1d156ad
Add an warning when the exception happens
jan-cerny Jun 7, 2021
e5be194
Add XCCDF 1.1 support
jan-cerny Jun 7, 2021
d5f1f13
Enable low-risk monkey-patching of the content filename.
matejak May 26, 2021
777e723
Improve feedback when unable to extract zipfiles.
matejak May 26, 2021
bc96da2
Decoupled shipped content location from its detection.
matejak May 26, 2021
47a39de
Add testing files
jan-cerny Jun 8, 2021
438873e
Various logging enhancements.
matejak May 26, 2021
b8bdaf7
Merge pull request #146 from matejak/small_fixes
jan-cerny Jun 9, 2021
306ac0d
Add a newline at the end of the file
jan-cerny Jun 9, 2021
cb7bd2c
Catch exception to prevent a fail
jan-cerny Jun 9, 2021
9266746
Allow empty string for tailoring path
jan-cerny Jun 9, 2021
5157660
Parse name space in a more elegant way
jan-cerny Jun 9, 2021
7b8ab3c
Fix PEP8 problems
jan-cerny Jun 9, 2021
2168d42
Refactor and improve readability of get_profiles()
jan-cerny Jun 9, 2021
99aec73
Fix Scrutinizer issue
jan-cerny Jun 9, 2021
ff1c74d
Merge pull request #147 from jan-cerny/scapcontenthandler
matejak Jun 9, 2021
49341dd
Merge pull request #149 from jan-cerny/new_test_content
matejak Jun 10, 2021
4a017d7
Fix some of the storage-related tests
poncovka Dec 9, 2020
03dfa61
Merge pull request #134 from poncovka/master-fix_storage_tests
matejak Jun 14, 2021
4392b51
Import mock from unittest, where it has been relocated.
matejak Jun 10, 2021
62f88b0
GH Actions: Single job for rhel8 branch
evgenyz Jun 14, 2021
64cec47
GH Actions: Single action for rhel9 branch
evgenyz Jun 14, 2021
998a82a
Merge pull request #155 from evgenyz/gh-actions-rhel8
matejak Jun 14, 2021
cf5a6a9
Merge pull request #156 from evgenyz/rhel9-branch
matejak Jun 14, 2021
f72c5de
Merge pull request #152 from matejak/update_mock_usage
jan-cerny Jun 15, 2021
2ea54ee
Slightly refactor handling of downloaded content.
matejak May 31, 2021
b0b7f00
Added a function for local content fetching.
matejak May 31, 2021
e3ff5b4
Enabled building images without sudo when possible.
matejak Jun 4, 2021
6632e53
Extracted rule data gathering.
matejak Jun 4, 2021
dfdc41c
Various improvements.
matejak Jun 4, 2021
3164e02
Introduced the Model class.
matejak Jun 4, 2021
4a85918
Fied GUI installs.
matejak Jun 7, 2021
3764f40
Improved code based on review feedback.
matejak Jun 9, 2021
ecdfff1
Pulled more functionality to the Model class.
matejak Jun 10, 2021
4574d94
Refactored handling of kickstart install abort.
matejak Jun 10, 2021
5c116dc
Rebranded the model class.
matejak Jun 10, 2021
cddc337
Merge RHEL8 changes to the RHEL9 branch.
matejak Jun 16, 2021
ff1cc83
Improve code style.
matejak Jun 16, 2021
76cf404
Cleaned up tests.
matejak Jun 17, 2021
670ba00
Increased robustness of content handling.
matejak Jun 17, 2021
9e0dd09
Update gating test configuration.
matejak Jun 17, 2021
1732af6
Come up with a better name for the certificate path.
matejak Jun 17, 2021
b7f2374
Merge pull request #148 from matejak/gathering_refactoring
jan-cerny Jun 17, 2021
0e590b5
Merge rhel8-branch into the rhel9_branch.
matejak Jun 18, 2021
b257de3
Merge pull request #158 from matejak/rhel9_port
jan-cerny Jun 22, 2021
92487c1
Merge the cpio module into the addon.
matejak Mar 11, 2021
4b70795
Fixed indentation and code style.
matejak Mar 11, 2021
56a1530
Merge pull request #140 from matejak/merge_cpio
jan-cerny Jun 25, 2021
2c3f9e2
Not only a Fedora addon.
matejak Jun 18, 2021
56ee09b
Dont use the tmp_path fixture.
matejak Jun 29, 2021
2dd3dee
Merge pull request #161 from matejak/no_tmp_path
jan-cerny Jun 30, 2021
e46c117
Flatten redundant Anaconda tasks
matejak Jun 18, 2021
e867ff3
Reuse the PolicyData structure.
matejak Jun 24, 2021
44134a2
Improve tests.
matejak Jun 29, 2021
7b4d355
Streamline the code
matejak Jun 30, 2021
f50d89a
Improve the ObtainedContent class
matejak Jun 30, 2021
2f49178
Fix an f-string that references incorrect variable
matejak Jun 30, 2021
f8cb96a
Fix conversion of fatal messages to strings
matejak Jun 30, 2021
5eacda3
Make the content prep task more robust and useful
matejak Jun 30, 2021
698e4c0
Remove the abort or continue question.
matejak Jun 29, 2021
8ca53c6
Disable attempts to fix invalid root passwords
matejak Jun 30, 2021
7046f39
Add missing initialization
matejak Jun 30, 2021
9d26843
Fix shadowing of _
matejak Jun 30, 2021
2e38bc8
Fix conversion of fatal messages to strings
matejak Jun 30, 2021
fc467d9
Streamline the code
matejak Jun 30, 2021
ea8fc0a
Fix an f-string that references incorrect variable
matejak Jun 30, 2021
16c6045
Merge pull request #159 from matejak/rhel9-polish
jan-cerny Jul 2, 2021
3e333ba
Merge pull request #162 from matejak/rhel8_fixes
jan-cerny Jul 7, 2021
3efcae4
Remove the spec file
matejak Jul 16, 2021
51ac407
Update translation support
matejak Jul 16, 2021
c1b61cd
Bump the release number
matejak Jul 7, 2021
8fc5472
Fix the packaging
matejak Jul 7, 2021
fe6cc3c
Bumped version after release
matejak Jul 30, 2021
7c7af33
Update translation template
matejak Jul 30, 2021
bcfd4da
Remove the RHEL8 spec file
matejak Jul 30, 2021
b7b11d6
Merge pull request #165 from matejak/post_120
jan-cerny Jul 30, 2021
d5b8a05
Merge pull request #163 from matejak/release_update
jan-cerny Jul 30, 2021
aeb0e2e
Improve logging
matejak Aug 2, 2021
b081e32
Save addon data when using local content
matejak Aug 2, 2021
fee170f
Refactor content identification
matejak Aug 3, 2021
1abc4e9
Refactor content identification
matejak Aug 3, 2021
a5c3e50
Merge pull request #167 from matejak/improv_8
jan-cerny Aug 3, 2021
3ef59fc
Merge pull request #166 from matejak/improv
jan-cerny Aug 3, 2021
3377a91
Fix archive handling in GUI installs
matejak Aug 9, 2021
191df32
Improve logging
matejak Aug 9, 2021
f7216cd
Merge pull request #168 from matejak/fix_two_step_download
jan-cerny Aug 9, 2021
6654fcd
Merge rhel8-branch into rhel9-branch
matejak Aug 20, 2021
56806b8
Represent unselected profile by an empty string
matejak Oct 15, 2021
c72b951
Fix bad destination for the parsed content fingerprint
matejak Oct 15, 2021
85d22eb
Merge pull request #176 from matejak/profile_not_none
jan-cerny Nov 2, 2021
983ee03
Merge pull request #177 from matejak/save_fingerprint
jan-cerny Nov 3, 2021
5627651
Merge pull request #170 from matejak/mmm
jan-cerny Nov 8, 2021
af840ee
Merge branch 'rhel9-branch'
matejak Nov 8, 2021
c6bfbd1
Fix import of the PackagesSelectionData class
poncovka Sep 23, 2021
bc45d77
Require the scanner for tests
matejak Nov 18, 2021
9b8457f
Use the PackagesSelection property
poncovka Nov 19, 2021
99cb67d
Fix usage of PackagesSelectionData in tests
matejak Nov 23, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yaml
Expand Up @@ -13,7 +13,7 @@ jobs:
image: fedora:35
steps:
- name: Install Deps
run: dnf install -y make anaconda openscap-python3 python3-cpio python3-mock python3-pytest python3-pycurl
run: dnf install -y make anaconda python3-pytest python3-pycurl openscap-scanner
- name: Checkout
uses: actions/checkout@v2
- name: Test
Expand Down
3 changes: 3 additions & 0 deletions AUTHORS
@@ -0,0 +1,3 @@
Vratislav Podzimek <v.podzimek@mykolab.com> - original author of the addon
Ignacio Vazquez-Abrams <ivazquez@ivazquez.net> - original author of the cpio module which has been merged into the addon
See https://github.com/OpenSCAP/oscap-anaconda-addon/graphs/contributors for an exhaustive list of other contributors.
13 changes: 10 additions & 3 deletions Makefile
@@ -1,15 +1,20 @@
NAME = oscap-anaconda-addon

VERSION = 3.0.0
VERSION = 0.35.0
Copy link
Contributor Author

@matejak matejak Nov 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose to use versions targetting Fedora in the format 0.<fedora version>.<patch version component>. Versions targetting RHEL then can use the classical major.minor.patch format, where neither major or minor correspond to RHEL versions - not every minor version of RHEL will get an OAA upgrade.


ADDON = org_fedora_oscap
TESTS = tests \
testing_files

DEFAULT_INSTALL_OF_PO_FILES ?= yes

PYVERSION = -3

TRANSLATIONS_DIR ?= po

FILES = $(ADDON) \
$(TESTS) \
data \
po \
COPYING \
Makefile \
Expand Down Expand Up @@ -86,7 +91,8 @@ potfile:
po-pull:
TEMP_DIR=$$(mktemp --tmpdir -d oscap-anaconda-addon-l10n-XXXXXXXXXX) && \
git clone --depth 1 -b $(GIT_L10N_BRANCH) -- $(L10N_REPOSITORY) $$TEMP_DIR && \
cp $$TEMP_DIR/$(OAA_PARENT_BRANCH)/*.po po/ && \
mkdir -p $(TRANSLATIONS_DIR) && \
cp $$TEMP_DIR/$(OAA_PARENT_BRANCH)/*.po $(TRANSLATIONS_DIR)/ && \
rm -rf $$TEMP_DIR

# This algorithm will make these steps:
Expand All @@ -104,6 +110,7 @@ update-pot:
$(MAKE) -C po potfile
TEMP_DIR=$$(mktemp --tmpdir -d oscap-anaconda-addon-l10n-XXXXXXXXXX) || exit 1 ; \
git clone --depth 1 -b $(GIT_L10N_BRANCH) -- $(L10N_REPOSITORY_RW) $$TEMP_DIR || exit 2 ; \
mkdir -p $$TEMP_DIR/$(OAA_PARENT_BRANCH) ; \
cp po/$(POTFILE_BASENAME) $$TEMP_DIR/$(OAA_PARENT_BRANCH)/ || exit 3 ; \
pushd $$TEMP_DIR/$(OAA_PARENT_BRANCH) ; \
git difftool --trust-exit-code -y -x "diff -u -I '^\"POT-Creation-Date: .*$$'" HEAD ./$(POTFILE_BASENAME) &>/dev/null ; \
Expand All @@ -128,7 +135,7 @@ container-test:
podman build --tag $(CONTAINER_NAME) --file tests/Dockerfile
podman run --volume .:/oscap-anaconda-addon:Z $(CONTAINER_NAME) make test

test: runpylint unittest
test: unittest runpylint

runpylint:
@echo "***Running pylint checks***"
Expand Down
18 changes: 12 additions & 6 deletions create_update_image.sh
Expand Up @@ -179,6 +179,8 @@ packages="
python3-cpio
python3-pycurl
oscap-anaconda-addon
xmlsec1
xmlsec1-openssl
"


Expand All @@ -204,8 +206,8 @@ install_addon_from_repo() {
else
install_po_files="DEFAULT_INSTALL_OF_PO_FILES=yes"
fi
# "copy files" to new root, sudo needed because we may overwrite files installed by rpm
sudo make install "$install_po_files" DESTDIR="${tmp_root}" >&2 || die "Failed to install the addon to $tmp_root."
# "copy files" to new root, sudo may be needed because we may overwrite files installed by rpm
$SUDO make install "$install_po_files" DESTDIR="${tmp_root}" >&2 || die "Failed to install the addon to $tmp_root."
}


Expand All @@ -216,12 +218,16 @@ create_image() {


cleanup() {
# cleanup, sudo needed because former RPM installs
sudo rm -rf "$tmp_root"
# cleanup, sudo may be needed because former RPM installs
$SUDO rm -rf "$tmp_root"
}


sudo true || die "Unable to get sudo working, bailing out."
if test $_arg_start_with_index -gt 1; then
SUDO=
else
SUDO=sudo
$SUDO true || die "Unable to get sudo working, bailing out."
fi

for (( action_index=_arg_start_with_index; action_index < ${#actions[*]}; action_index++ )) do
"${actions[$action_index]}"
Expand Down
69 changes: 61 additions & 8 deletions docs/manual/developer_guide.adoc
Expand Up @@ -15,8 +15,7 @@ What do you need to test your changes?

1. Clone addon repository & use required branch & change code.
2. Build addon update image.
3. Serve image using local HTTP server.
4. Load system with update image.
3. Run a VM using reanaconda.


=== Clone repository & use required branch
Expand Down Expand Up @@ -51,8 +50,66 @@ or use the `lsinitrd` command, which is part of the `dracut` package on RHEL and
lsinitrd update.img
----

=== Use reanaconda

=== Serve image using HTTP server
The `reanaconda` script prepares and starts a VM with the update image.
You can get it from the repository at
https://github.com/rhinstaller/devel-tools/tree/master/reanaconda

Prepare the VM:

----
./reanaconda.py prime --sensible --tree http://ftp.fi.muni.cz/pub/linux/fedora/linux/releases/34/Everything/x86_64/os
----

After the script terminates, provide the update image:

----
./reanaconda.py updates path/to/updates.img
----

The VM should be shown in a QEMU window, which you can play with and you can
close it any time.

If you run a Fedora VM, there won't be `scap-security-guide` content available,
so you will have to serve the content from your host machine using a HTTP
server. For example, you can provide your local SSG build. In a new terminal,
run:

----
cd ~/work/git/scap-security-guide/build
python3 -m http.server
----

And then, in the OSCAP Anaconda Addon user interface, enter the URL. Your host
is visible from your guest at `10.0.2.2`. For example:

----
http://10.0.2.2:8000/ssg-fedora-ds.xml
----

Watch the console, as the VM is supposed to download the update image, and the Python server should output the corresponding HTTP request:

`<IP> - - [<date>] "GET /update.img HTTP/1.1" 200 -`

200 is the OK request status.

There is a cleanup step, but you don't have to run it if you only want to
restart the VM with a new image:

----
./reanaconda.py cleanup
----

=== Further introspection of Anaconda

After reaching the Anaconda GUI, you can switch into another VT and check out that the update went OK by examining the files on the disc.
You can also debug Anaconda in a sophisticated way - as of 04/2018, switching to tty1 brought you to a TMUX session with windows attached to various processes.
There is also an official https://fedoraproject.org/wiki/How_to_debug_installation_problems[how-to-debug documentation] though.

=== Older method without reanaconda

==== Serve image using HTTP server

You don't need public HTTP server or setup Apache.
You can use simple python HTTP server - it serves all files in you current directory.
Expand All @@ -64,7 +121,7 @@ python3 -m http.server
**Setup your firewall rules correctly to make webserver port accessible from virtual machine.**


=== Load system with update image
==== Load system with update image

If you want to load your changes to anaconda, you have to setup boot options correctly.
You have two ways how to setup it:
Expand Down Expand Up @@ -99,11 +156,7 @@ Watch the console, as the VM is supposed to download the update image, and the P
200 is the OK request status.


=== Further introspection of Anaconda

After reaching the Anaconda GUI, you can switch into another VT and check out that the update went OK by examining the files on the disc.
You can also debug Anaconda in a sophisticated way - as of 04/2018, switching to tty1 brought you to a TMUX session with windows attached to various processes.
There is also an official https://fedoraproject.org/wiki/How_to_debug_installation_problems[how-to-debug documentation] though.


== Available make commands
Expand Down