Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

modularize features #52

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
df13d6e
added placeholder for features on gen_* scripts
timkenhan Sep 13, 2023
d103821
improved Makefile
timkenhan Sep 13, 2023
2b73ed8
added genkernel.conf to the generation code
timkenhan Sep 15, 2023
d3241e9
added genkernel.8 manpage to the generation code
timkenhan Sep 17, 2023
c2bc82c
added genkernel.8 manpage to the generation code
timkenhan Sep 17, 2023
c47ea1b
started implementing the automation
timkenhan Sep 17, 2023
49935eb
moving forward with the automation
timkenhan Sep 18, 2023
8c428a1
the generation code works!
timkenhan Sep 18, 2023
f8d94b7
the generation code works!
timkenhan Sep 18, 2023
5453025
moved the rest of the modular features
timkenhan Sep 19, 2023
2fab801
added manpage to make install
timkenhan Sep 19, 2023
7a5338a
more safety on generation code
timkenhan Sep 19, 2023
2d21960
some cleanup on generation output
timkenhan Sep 20, 2023
abdb880
improved makefile
timkenhan Sep 26, 2023
7828ec4
improved makefiles for the pesky implicit rules
timkenhan Oct 4, 2023
507607b
updated the README
timkenhan Oct 10, 2023
a97c810
massive cleanup on makefiles
timkenhan Oct 11, 2023
6d9ed13
added dependency on software.sh
timkenhan Oct 11, 2023
4410c5e
changed build output directory
timkenhan Oct 11, 2023
74c8f45
separates software.sh package versions
timkenhan Oct 11, 2023
9609e5e
prevent failures on missing distfiles
timkenhan Oct 11, 2023
fab8f10
added missing worker_modules
timkenhan Oct 11, 2023
4f00802
added missing gkbuilds
timkenhan Oct 11, 2023
028988f
restructured feature files; improved Makefile
timkenhan Oct 17, 2023
76b76d3
Merge branch 'master' into modularize-features
timkenhan Nov 2, 2023
a280e6a
added missing build-config
timkenhan Nov 6, 2023
3b126e6
Merge branch 'gentoo:master' into modularize-features
timkenhan Jan 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
/genkernel-*.tar.bz2
/ChangeLog
/arch/*/generated-config
/out
165 changes: 158 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,31 +1,42 @@
#PACKAGE_VERSION = $(shell /bin/grep -F -- GK_V= genkernel | sed "s/.*GK_V='\([^']\+\)'/\1/")
PACKAGE_VERSION = $(shell git describe --tags |sed 's,^v,,g')
distdir = genkernel-$(PACKAGE_VERSION)
MANPAGE = genkernel.8
MANPAGE := genkernel.8
# Add off-Git/generated files here that need to be shipped with releases
EXTRA_DIST = $(MANPAGE) ChangeLog $(KCONF)

default: kconfig man

# First argument in the override file
# Second argument is the base file
BASE_KCONF = defaults/kernel-generic-config
ARCH_KCONF = $(wildcard arch/*/arch-config)
GENERATED_KCONF = $(subst arch-,generated-,$(ARCH_KCONF))
KCONF = $(GENERATED_KCONF)

PREFIX = /usr/local
BINDIR = $(PREFIX)/bin
ifeq ($(PREFIX), /usr)
SYSCONFDIR = /etc
else
SYSCONFDIR = $(PREFIX)/etc
endif
MANDIR = $(PREFIX)/share/man


all: out/genkernel man kconfig

debug:
@echo "ARCH_KCONF=$(ARCH_KCONF)"
@echo "GENERATED_KCONF=$(GENERATED_KCONF)"

kconfig: $(GENERATED_KCONF)
man: $(MANPAGE)
man: $(foreach file, $(MANPAGE), out/$(file))
timkenhan marked this conversation as resolved.
Show resolved Hide resolved

ChangeLog:
git log >$@

clean:
rm -f $(EXTRA_DIST)
rm -rf out
timkenhan marked this conversation as resolved.
Show resolved Hide resolved

check-git-repository:
ifneq ($(UNCLEAN),1)
Expand Down Expand Up @@ -56,9 +67,9 @@ distclean: clean
perl merge.pl $< $(BASE_KCONF) | sort > $@ ; \
fi ;

%.8: doc/%.8.txt doc/asciidoc.conf Makefile genkernel
a2x --conf-file=doc/asciidoc.conf --attribute="genkernelversion=$(PACKAGE_VERSION)" \
--format=manpage -D . "$<"
out/%.8: doc/%.8.txt doc/asciidoc.conf Makefile out/doc/genkernel.8.txt
a2x --conf-file=doc/asciidoc.conf \
--format=manpage -D out "out/$<"

verify-doc: doc/genkernel.8.txt
@rm -f faildoc ; \
Expand Down Expand Up @@ -99,3 +110,143 @@ verify-shellscripts-initramfs:
--severity error \
defaults/linuxrc \
defaults/initrd.scripts

out/%/:
install -d $@

out/temp/%: out/temp/
echo > $@

out/build-config: out/temp/genkernel_conf out/temp/man_genkernel_8 out/temp/parse_cmdline \
timkenhan marked this conversation as resolved.
Show resolved Hide resolved
out/temp/longusage out/temp/append_base_layout out/temp/create_initramfs \
out/temp/initramfs_append_func out/temp/determine_real_args
ifdef GK_FEATURES
awk -f compile_features.awk $(foreach feature,${GK_FEATURES},features/$(feature))
timkenhan marked this conversation as resolved.
Show resolved Hide resolved
endif
echo ${PREFIX} > out/PREFIX
echo ${BINDIR} > out/BINDIR
echo ${SYSCONFDIR} > out/SYSCONFDIR
echo ${MANDIR} > out/MANDIR
touch out/build-config

out/genkernel.conf: out/build-config
cat genkernel.conf | sed \
-e '/#BEGIN FEATURES genkernel_conf/ r out/temp/genkernel_conf' \
> out/genkernel.conf
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't trigger properly on updates.
If I edit something in genkernel.conf and run make again, out/genkernel.conf will not be regenerated.

It's also unclear why it depends on out/build-config when it does NOT consume that file at all.

out/genkernel.conf: genkernel.conf out/temp/genkernel_conf
    cat $< | sed -e '/#BEGIN FEATURES genkernel_conf/ r out/temp/genkernel_conf' >$@

The same issue exists for many of your other targets.

I suggest creating a helper script that supports every transformation you've coded into the Makefile, and then simplifying the makefile:

out/genkernel.conf: genkernel.conf out/build-config $(INTERMEDIATE_DEPS) helper.sh
  ./helper.sh < $< > $@

and then you can see how it's easy to convert that rule further (this is a little harder without changing the layout to have the source files in a distinct directory).

out/%.conf: %.conf out/build-config $(INTERMEDIATE_DEPS) helper.sh
  ./helper.sh < $< > $@

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created the build-config despite not being directly consumed because i was having problem getting the proper dependencies to register for preventing rebuilds. I thought about having separate dependencies for the temp files, but they are all processed at once within the awk script.


out/doc/genkernel.8.txt: out/build-config out/doc/
cat doc/genkernel.8.txt | sed \
-e '/\/\/ BEGIN FEATURES man_genkernel_8/ r out/temp/man_genkernel_8' \
> out/doc/genkernel.8.txt

out/gen_cmdline.sh: out/build-config
cat gen_cmdline.sh | sed \
-e '/#BEGIN FEATURES parse_cmdline()/ r out/temp/parse_cmdline' \
-e '/#BEGIN FEATURES longusage()/ r out/temp/longusage' \
> out/gen_cmdline.sh

out/gen_initramfs.sh: out/build-config
cat gen_initramfs.sh | sed \
-e '/#BEGIN FEATURES append_base_layout()/ r out/temp/append_base_layout' \
-e '/#BEGIN FEATURES create_initramfs()/ r out/temp/create_initramfs' \
-e '/#BEGIN FEATURES initramfs_append_func/ r out/temp/initramfs_append_func' \
> out/gen_initramfs.sh

out/gen_determineargs.sh: out/build-config
cat gen_determineargs.sh | sed \
-e '/#BEGIN FEATURES determine_real_args()/ r out/temp/determine_real_args' \
> out/gen_determineargs.sh

out/software.sh:
cat defaults/software.sh | sed \
-e "s:VERSION_BCACHE_TOOLS:${VERSION_BCACHE_TOOLS}:"\
-e "s:VERSION_BOOST:${VERSION_BOOST}:"\
-e "s:VERSION_BTRFS_PROGS:${VERSION_BTRFS_PROGS}:"\
-e "s:VERSION_BUSYBOX:${VERSION_BUSYBOX}:"\
-e "s:VERSION_COREUTILS:${VERSION_COREUTILS}:"\
-e "s:VERSION_CRYPTSETUP:${VERSION_CRYPTSETUP}:"\
-e "s:VERSION_DMRAID:${VERSION_DMRAID}:"\
-e "s:VERSION_DROPBEAR:${VERSION_DROPBEAR}:"\
-e "s:VERSION_EUDEV:${VERSION_EUDEV}:"\
-e "s:VERSION_EXPAT:${VERSION_EXPAT}:"\
-e "s:VERSION_E2FSPROGS:${VERSION_E2FSPROGS}:"\
-e "s:VERSION_FUSE:${VERSION_FUSE}:"\
-e "s:VERSION_GPG:${VERSION_GPG}:"\
-e "s:VERSION_HWIDS:${VERSION_HWIDS}:"\
-e "s:VERSION_ISCSI:${VERSION_ISCSI}:"\
-e "s:VERSION_JSON_C:${VERSION_JSON_C}:"\
-e "s:VERSION_KMOD:${VERSION_KMOD}:"\
-e "s:VERSION_LIBAIO:${VERSION_LIBAIO}:"\
-e "s:VERSION_LIBGCRYPT:${VERSION_LIBGCRYPT}:"\
-e "s:VERSION_LIBGPGERROR:${VERSION_LIBGPGERROR}:"\
-e "s:VERSION_LIBXCRYPT:${VERSION_LIBXCRYPT}:"\
-e "s:VERSION_LVM:${VERSION_LVM}:"\
-e "s:VERSION_LZO:${VERSION_LZO}:"\
-e "s:VERSION_MDADM:${VERSION_MDADM}:"\
-e "s:VERSION_MULTIPATH_TOOLS:${VERSION_MULTIPATH_TOOLS}:"\
-e "s:VERSION_POPT:${VERSION_POPT}:"\
-e "s:VERSION_STRACE:${VERSION_STRACE}:"\
-e "s:VERSION_THIN_PROVISIONING_TOOLS:${VERSION_THIN_PROVISIONING_TOOLS}:"\
-e "s:VERSION_UNIONFS_FUSE:${VERSION_UNIONFS_FUSE}:"\
-e "s:VERSION_USERSPACE_RCU:${VERSION_USERSPACE_RCU}:"\
-e "s:VERSION_UTIL_LINUX:${VERSION_UTIL_LINUX}:"\
-e "s:VERSION_XFSPROGS:${VERSION_XFSPROGS}:"\
-e "s:VERSION_XZ:${VERSION_XZ}:"\
-e "s:VERSION_ZLIB:${VERSION_ZLIB}:"\
-e "s:VERSION_ZSTD:${VERSION_ZSTD}:"\
> out/software.sh
timkenhan marked this conversation as resolved.
Show resolved Hide resolved

out/genkernel: out/genkernel.conf out/gen_cmdline.sh \
out/gen_initramfs.sh out/gen_determineargs.sh out/software.sh

cp gen_arch.sh out/gen_arch.sh
cp gen_bootloader.sh out/gen_bootloader.sh
cp gen_compile.sh out/gen_compile.sh
cp gen_configkernel.sh out/gen_configkernel.sh
cp gen_funcs.sh out/gen_funcs.sh
cp gen_moddeps.sh out/gen_moddeps.sh
cp gen_package.sh out/gen_package.sh
cp gen_worker.sh out/gen_worker.sh
cp path_expander.py out/path_expander.py
cp genkernel out/genkernel
timkenhan marked this conversation as resolved.
Show resolved Hide resolved


install: PREFIX := $(file <out/PREFIX)
install: BINDIR := $(file <out/BINDIR)
install: SYSCONFDIR := $(file <out/SYSCONFDIR)
install: MANDIR := $(file <out/MANDIR)
install: all
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing deps here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you elaborate on this? i've put the deps on top of file

install -d $(DESTDIR)/$(SYSCONFDIR)
install -m 644 out/genkernel.conf $(DESTDIR)/$(SYSCONFDIR)/

install -d $(DESTDIR)/$(BINDIR)
install -m 755 out/genkernel $(DESTDIR)/$(BINDIR)/

install -d $(DESTDIR)/$(PREFIX)/share/genkernel

cp -rp arch $(DESTDIR)/$(PREFIX)/share/genkernel/
cp -rp defaults $(DESTDIR)/$(PREFIX)/share/genkernel/
cp -rp modules $(DESTDIR)/$(PREFIX)/share/genkernel/
cp -rp netboot $(DESTDIR)/$(PREFIX)/share/genkernel/
cp -rp patches $(DESTDIR)/$(PREFIX)/share/genkernel/

install -m 755 out/gen_arch.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_bootloader.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_cmdline.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_compile.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_configkernel.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_determineargs.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_funcs.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_initramfs.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_moddeps.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_package.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/gen_worker.sh $(DESTDIR)/$(PREFIX)/share/genkernel
install -m 755 out/path_expander.py $(DESTDIR)/$(PREFIX)/share/genkernel
timkenhan marked this conversation as resolved.
Show resolved Hide resolved

install out/software.sh $(DESTDIR)/$(PREFIX)/share/genkernel/defaults

install -d $(DESTDIR)/$(MANDIR)
install out/genkernel.8 $(DESTDIR)/$(MANDIR)/man8

# install -d $(DESTDIR)/var/lib/genkernel/src
# install -m 644 tarballs/* $(DESTDIR)/var/lib/genkernel/src/
15 changes: 15 additions & 0 deletions README
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,23 @@ things mentioned above.
DEPENDENCIES:
- GCC and binutils
- GNU Bash
- GNU Awk
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it really need gawk, or just any posix awk? Does mawk work?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it would work with any awk. I've tried it with mawk and it works. Should I update it accordingly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes please

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

- GNU Make
- PaX Utilities
- Autoconf and Automake
- Asciidoc
- Dependencies of each feature included

INSTALLATION:
Build with ``make PREFIX=<prefix> GK_FEATURES=<features>''
<features> is a space-separated containing files in features/ directory,
e.g. GK_FEATURES="luks, lvm, plymouth".
Optionally:
- BINDIR: defaults to "$PREFIX/bin"
- SYSCONFDIR: defaults to "/etc" for PREFIX=/usr or "$PREFIX/etc" otherwise
- MANDIR: defaults to "$PREFIX/share/man"

Then install with ``make install'', optionally with DESTDIR

USAGE:
See https://wiki.gentoo.org/wiki/Genkernel or
Expand Down
28 changes: 28 additions & 0 deletions compile_features.awk
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
function ltrim(s) { sub(/^[ \t\r\n]+/, "", s); return s }
timkenhan marked this conversation as resolved.
Show resolved Hide resolved
function rtrim(s) { sub(/[ \t\r\n]+$/, "", s); return s }
function trim(s) { return rtrim(ltrim(s)); }

BEGIN { FS="\n\n-----\n\n"; RS="\0" }

{
for(i=1; i<=NF; ++i) {
S = trim($i)
split(S, A, "\n")
L = length(A)
S_begin = A[1]
S_end = A[L]
if (substr(S_begin, 0, 7) != "%begin " || substr(S_end, 0, 5) != "%end ") {
print "invalid section header/footer in: "FILENAME
exit 1
}
section_name = substr(S_begin, 8)
if (substr(S_end, 6) != section_name) {
print "invalid section name in: "FILENAME
exit 1
}
filename="out/temp/"section_name
for(j=2; j<L; ++j)
print A[j] >> filename
print "" >> filename
}
}