stage1 handling and configuration #2035

krnowak · 2016-01-26T16:12:41Z

rewrite stage1hash.go
- replace --stage1-image flag with --stage1-{url,path,name,hash,from-dir}
add stage1 configuration
- defines name, version and location of a stage1 image
  - name and version are used to search for the image in the image store
  - location is used for fetching the image if it is not in the store
extend paths configuration
- add stage1 images directory path
  - used together with the --stage1-from-dir flag
add the stage1 loading test
- it tests all the --stage1-* flags
- it tests the configuration
add some features to the test aci server, required by the stage1 loading test
- add caching (etag handling)
- add http connections
add the stub stage1 image fot the stage1 loading test
- it does almost nothing
- might be useful in other tests that check something between running rkt and actual execution of the coreos.com/rkt/stage1/run entrypoint.
add some per-image flags for specifying the type of the image reference
- likely to be dropped, be a part of a different PR
- wrote it long time ago (when refactoring image handling)
- forgot to test it

alban · 2016-01-26T16:18:50Z

CHANGELOG.md


 #### New features and UX changes

 - Add pod creation and start times to `rkt list` and `rkt status` ([#2030](https://github.com/coreos/rkt/pull/2030)). See [`rkt list`](https://github.com/coreos/rkt/blob/master/Documentation/subcommands/list.md) and [`rkt status`](https://github.com/coreos/rkt/blob/master/Documentation/subcommands/status.md) documentation.
+- Added configuration for stage1 image. User can drop a configuration file to `/etc/rkt/stage1.d` (or to `stage1.d` in user configuration directory) to tell rkt to use different name, version and location of stage1 image instead of build-time defaults ([#1977](https://github.com/coreos/rkt/pull/1977)).


s/1977/2035/g

ditto below

krnowak · 2016-02-01T12:08:40Z

Updated.

krnowak · 2016-02-01T15:11:41Z

Oh god, conflicts again.

iaguis · 2016-02-02T09:33:46Z

Documentation/configuration.md

+##### Command line flags
+
+The `data` field can be overridden with the `--dir` flag.
+The `stage1-images` cannot be overridden with the command line flag.


iaguis · 2016-02-02T10:51:40Z

CHANGELOG.md

 #### Note for packagers

 With this release, `rkt` RPM/dpkg packages should have the following updates:

 - Pass `--enable-tpm=no` to configure script, if `rkt` should not use TPM.
+- Use the `--with-default-stage1-images-directory` configure flag, if the default is not satisfiable and install the built stage1 images there.


s/if the default is not satisfiable/if the default is not acceptable,/

iaguis · 2016-02-02T11:18:21Z

https://github.com/coreos/rkt/blob/master/Documentation/subcommands/run.md and https://github.com/coreos/rkt/blob/master/Documentation/subcommands/prepare.md need update.

iaguis · 2016-02-02T11:32:00Z

LGTM after the doc fixes and on green

This extends the "override semantics" section. Also, it rewords or fixes other parts of the documentation.

Some of the configured values can be overridden with command line flags, so it would be nice to mention which ones for each kind.

build: Use a better wording in configure.ac (conflicting -> mutually exclusive), reformat ldflags for clarity in rkt.mk config: Document some public structs, remove a useless call to os.TempDir(), make sure that data directory is an absolute path, fix a bug in toArray functional tests: Minor code shuffling and renaming in test aci server. image: print a message when using stage1 image from a hash, fix a null pointer dereference when using finder

This setting will be later used by --stage1-from-dir flag.

It replaces the --stage1-image flag with five mutually-exclusive flags: --stage1-url, --stage1-path, --stage1-name, --stage1-hash and --stage1-from-dir. The flags tell rkt to use stage1 image, respectively, from a URL, from a path, in the store under the name, in the store with the hash, from a file in the default stage1 images directory. In case none of the above flags were provided, rkt takes stage1 and paths configuration and build-time defaults to decide for which name and version of the stage1 image it should search for in the image store and, if that fails, from which location it should fetch the image. This introduces a new configure flag for defining the path to the default stage1 images directory.

…nder http This will be used in the stage1 loading test to test stage1 loading either from a configuration with an http location defined or from the --stage1-url flag with an http url.

This will be used in the stage1 loading test to make sure that rkt reuses a stage1 image from the store instead of fetching it from the remote location again.

…truct The list of options will grow, which will make the NewServer function even more awkward (with 5 parameters or so). So this commit puts all the configuration into the ServerSetup struct and provides a function for getting a default setup, which can be modified before passing it to NewServer function.

…ndom one Choosing a fixed port will make the server to listen on port 80 or 443, depending on whether it uses http or https protocol. This also replaces a hack for running a discovery server with a proper solution, which is replacing a default listener with our own.

This implementation is actually so minimal, that it does not even bother with running stage2. We can use it regardless of how the project was configured. This stage1 image could also be used in other tests that do not involve checking things happening inside stage1 or stage2. This will be used in stage1 loading test.

The stage1 loading test will use stub stage1 image for its checks. One of them is testing whether rkt falls back to fetching stage1 image from the same directory the rkt binary is in. To do this, we need to put the stub stage1 image in the same directory as rkt. To avoid a confusion from packagers and users, we do it backwards - we put a symlink to the rkt binary in the directory where stub stage1 image is. In the same directory we also put symlinks to all the flavors of the stage1 image, so the tests should work as they did so far.

It tests loading stage1 from the configuration and from the flags. Docker cases are not tested at all - we need a docker test server first.

iaguis · 2016-02-02T13:35:52Z

LGTM

stage1 handling and configuration

krnowak mentioned this pull request Jan 26, 2016

stage1 handling and configuration #1977

Closed

alban reviewed Jan 26, 2016
View reviewed changes

krnowak force-pushed the krnowak/stage1-config branch from fcc2ac3 to 541bf94 Compare February 1, 2016 11:12

iaguis added kind/enhancement component/stage1 area/usability kind/design priority/P1 labels Feb 1, 2016

iaguis added this to the v1.0.0 milestone Feb 1, 2016

krnowak force-pushed the krnowak/stage1-config branch from 541bf94 to d16a704 Compare February 1, 2016 12:07

iaguis added the needs/review label Feb 1, 2016

krnowak force-pushed the krnowak/stage1-config branch from d16a704 to e8aed00 Compare February 1, 2016 12:19

krnowak force-pushed the krnowak/stage1-config branch 2 times, most recently from 69899ef to aa16d92 Compare February 2, 2016 08:21

iaguis reviewed Feb 2, 2016
View reviewed changes

krnowak force-pushed the krnowak/stage1-config branch from aa16d92 to 3771968 Compare February 2, 2016 11:10

krnowak force-pushed the krnowak/stage1-config branch from 3771968 to 0fbb13c Compare February 2, 2016 11:34

krnowak added 6 commits February 2, 2016 14:05

docs: Extend the paths configuration.

ab1777d

This extends the "override semantics" section. Also, it rewords or fixes other parts of the documentation.

docs: Add command line sections in documentation about configuration

ef8f733

Some of the configured values can be overridden with command line flags, so it would be nice to mention which ones for each kind.

config, docs: Add default stage1 images directory to the paths config

205e011

This setting will be later used by --stage1-from-dir flag.

config: Add some configuration for the default stage1 image

adb0a67

krnowak force-pushed the krnowak/stage1-config branch from 0fbb13c to c048bda Compare February 2, 2016 13:23

krnowak added 10 commits February 2, 2016 14:28

docs: Use one of the new stage1 flags.

d9b3f9b

bash: Update autocomplete with new flags

9c61e02

functional tests: Add an option to test server to make it available u…

be46fb4

…nder http This will be used in the stage1 loading test to test stage1 loading either from a configuration with an http location defined or from the --stage1-url flag with an http url.

functional tests: Add caching support to test aci server

d9d89ef

This will be used in the stage1 loading test to make sure that rkt reuses a stage1 image from the store instead of fetching it from the remote location again.

functional tests: New test for loading stage1 image

b91cd7d

It tests loading stage1 from the configuration and from the flags. Docker cases are not tested at all - we need a docker test server first.

CHANGELOG.md: Mention stage1 loading work

deeb43f

krnowak force-pushed the krnowak/stage1-config branch from c048bda to deeb43f Compare February 2, 2016 13:28

krnowak mentioned this pull request Feb 2, 2016

tests: add more tests for rkt trust and signed images #2070

Open

iaguis added a commit that referenced this pull request Feb 2, 2016

Merge pull request #2035 from kinvolk/krnowak/stage1-config

25ddde5

stage1 handling and configuration

iaguis merged commit 25ddde5 into rkt:master Feb 2, 2016

iaguis mentioned this pull request Feb 2, 2016

help: '--stage1-image' option is confusing #1600

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stage1 handling and configuration #2035

stage1 handling and configuration #2035

krnowak commented Jan 26, 2016

alban Jan 26, 2016

alban Jan 26, 2016

krnowak commented Feb 1, 2016

krnowak commented Feb 1, 2016

iaguis Feb 2, 2016

iaguis Feb 2, 2016

iaguis commented Feb 2, 2016

iaguis commented Feb 2, 2016

iaguis commented Feb 2, 2016

stage1 handling and configuration #2035

stage1 handling and configuration #2035

Conversation

krnowak commented Jan 26, 2016

alban Jan 26, 2016

Choose a reason for hiding this comment

alban Jan 26, 2016

Choose a reason for hiding this comment

krnowak commented Feb 1, 2016

krnowak commented Feb 1, 2016

iaguis Feb 2, 2016

Choose a reason for hiding this comment

iaguis Feb 2, 2016

Choose a reason for hiding this comment

iaguis commented Feb 2, 2016

iaguis commented Feb 2, 2016

iaguis commented Feb 2, 2016