-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Finalise interface between composer and image definitions #3444
Finalise interface between composer and image definitions #3444
Commits on May 30, 2023
-
distro: inline initializeManifest
Inline the initializeManifest() function so we can start simplifying the PackageSets() and Manifest() (the two callers) separately.
Configuration menu - View commit details
-
Copy full SHA for 1914131 - Browse repository at this point
Copy the full SHA 1914131View commit details -
Configuration menu - View commit details
-
Copy full SHA for abd68dc - Browse repository at this point
Copy the full SHA abd68dcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 730840c - Browse repository at this point
Copy the full SHA 730840cView commit details -
distro: simplify checkOptions() arguments
Make checkOptions() take the whole blueprint and options. There is no need to pass in the resolved containers separately since we only care whether there are any containers defined for image types that don't support them.
Configuration menu - View commit details
-
Copy full SHA for c3a9654 - Browse repository at this point
Copy the full SHA c3a9654View commit details -
subscription: new package for subscription options
Move the subscription options from distro to its own package. Now we can import the manifest package into the distro package (instead of the other way around) so we can work with the manifest.Manifest type in distro.
Configuration menu - View commit details
-
Copy full SHA for 54ec501 - Browse repository at this point
Copy the full SHA 54ec501View commit details -
distro: remove architecture names and use platform enum
Removing the dependence of the manifest package on the distro package to import manifest into distro. Wherever arch names are needed, we use the enums from the platform package instead.
Configuration menu - View commit details
-
Copy full SHA for 2850288 - Browse repository at this point
Copy the full SHA 2850288View commit details -
manifest: serialize to OSBuildManifest instead of distro.Manifest
Copy the Marshal and Unmarshal functions from distro.Manifest to manifest.OSBuildManifest to keep the same behaviour. The Version() function isn't used, so let's drop it.
Configuration menu - View commit details
-
Copy full SHA for b624d36 - Browse repository at this point
Copy the full SHA b624d36View commit details -
distro: delete distro.Manifest type
Delete the distro.Manifest type and its tests. Change all occurrences of distro.Manifest to manifest.OSBuildManifest.
Configuration menu - View commit details
-
Copy full SHA for 034a7bb - Browse repository at this point
Copy the full SHA 034a7bbView commit details -
manifest: expand the manifest package doc
Expand the manifest package documentation and move it to the manifest.go file.
Configuration menu - View commit details
-
Copy full SHA for 3038403 - Browse repository at this point
Copy the full SHA 3038403View commit details -
container: create a SourceSpec for the resolve parameters
Create a struct to conveniently hold the source parameters for a container resolve request and use it in the Add() function of the resolver.
Configuration menu - View commit details
-
Copy full SHA for 4af3641 - Browse repository at this point
Copy the full SHA 4af3641View commit details -
ostree: rename RequestParams to SourceSpec
Same as with the container SourceSpec, the struct specifies the required information to resolve an ostree commit from a source (URL, ref, and optional parent). Renaming for consistency.
Configuration menu - View commit details
-
Copy full SHA for d7e6ad2 - Browse repository at this point
Copy the full SHA d7e6ad2View commit details -
Configuration menu - View commit details
-
Copy full SHA for f6839f9 - Browse repository at this point
Copy the full SHA f6839f9View commit details -
pipeline: expand interface to return source specs
Add two new methods to the interface for returning container source specifications and ostree commit source specifications respectively. These are analogous to the package set chains; they are unresolved source specifications that can be used with the appropriate resolver to create a full specification for a manifest. Only the Base pipeline implements the base empty functions for these for now.
Configuration menu - View commit details
-
Copy full SHA for 53e2512 - Browse repository at this point
Copy the full SHA 53e2512View commit details -
rhsm: move FactsImageOptions to the rhsm/facts package
Move the FactsImageOptions from distro to the new rhsm/facts package. At the same time define the values we use as an enum, including the "test-manifest" value. Though the values don't really matter, the test value is defined first so it takes the 0 value, which feels nicer conceptually. The field in the distro.ImageOptions is changed to be a pointer to allow for nil values.
Configuration menu - View commit details
-
Copy full SHA for 2b2e10e - Browse repository at this point
Copy the full SHA 2b2e10eView commit details -
ostree: move OSTreeImageOptions to the ostree package
Move the ostree image options to the ostree package and rename the type to ImageOptions (ostree.ImageOptions).
Configuration menu - View commit details
-
Copy full SHA for 658c62f - Browse repository at this point
Copy the full SHA 658c62fView commit details -
Make Manifest() return manifest.Manifest
Return manifest.Manifest from the Manifest() function without serializing. The caller then has to call the manifest.Serialize() function using the depsolved packages. This moves towards changing the order of actions required to generate a manifest. With this change, the manifest creation and depsolving can be done independently, but this still requires instantiating the manifest object twice (InstantiateManifest() is called in PackageSets() and Manifest()), which we don't want to have to do.
Configuration menu - View commit details
-
Copy full SHA for f3da509 - Browse repository at this point
Copy the full SHA f3da509View commit details -
manifest: attach unresolved Content to Manifest struct
Add Content to the Manifest struct to represent unresolved packages, container, and ostree commit specs that the caller should resolve before serialising the manifest.
Configuration menu - View commit details
-
Copy full SHA for 3fc6a11 - Browse repository at this point
Copy the full SHA 3fc6a11View commit details -
manifest: document Pipeline interface
Docstrings for the Manifest.pipelines field and the Pipeline type itself.
Configuration menu - View commit details
-
Copy full SHA for 388f866 - Browse repository at this point
Copy the full SHA 388f866View commit details -
distro: pass entire Blueprint to Manifest()
Pass the entire Blueprint to Manifest() instead of just the Customizations. The goal is to combine the functionality of the ImageType.PackageSets() and ImageType.Manifest() methods into one call.
Configuration menu - View commit details
-
Copy full SHA for cfd0c85 - Browse repository at this point
Copy the full SHA cfd0c85View commit details -
distro: copy PackageSets() functionality into Manifest()
Copy the functionality of the ImageType.PackageSets() methods into ImageType.Manifest() for each distro. The Manifest() method now collects all package sets and repositories from the blueprint and image type and after generating the Manifest instance, calls the GetPackageSetChains() method to attach the computed package sets to the Manifest before returning it. The package sets in the call are now renamed to staticPackageSets to differentiate from the dynamic (computed) package sets that are affected by the manifest generation.
Configuration menu - View commit details
-
Copy full SHA for b2178f8 - Browse repository at this point
Copy the full SHA b2178f8View commit details -
cmd: remove usage of ImageType.PackageSets()
Use the new manifest generation procedure in the cmd line tools. The new procedure doesn't rely on ImageType.PackageSets() to compute the packages for the depsolving. Instead, it calls Manifest() and depsolves the packages attached to the returned object (manifest.Content.PackageSets).
Configuration menu - View commit details
-
Copy full SHA for c0c9d27 - Browse repository at this point
Copy the full SHA c0c9d27View commit details -
weldr: remove usage of ImageType.PackageSets()
Use the new manifest generation procedure in the Weldr API. Updated test distro to include the same packages from the PackageSets() method in the Manifest.Content.PackageSets.
Configuration menu - View commit details
-
Copy full SHA for 51a6d44 - Browse repository at this point
Copy the full SHA 51a6d44View commit details -
distro: attach payload repositories to workload package sets
The merging of payload repositories into the os pipeline had the unwanted side effect of using the payload repos for the first depsolve in the os chain when instead they should only be used for the second (the depsolve for the blueprint or workload packages). This wasn't an issue before because we didn't do the merging in the PackageSets() function, but now we rely on the Manifest() function for that functionality instead. Before the merging of the two functions, the PackageSets() function did not merge repositories and the repository-to-package-set mapping was maintained correctly, but the merging was needed in the Manifest() function so that rpm stage options were generated for all repositories. With this change, we are removing the merging so that the mapping is maintained, and will fix the rpm stage option generation in the pipeline generator.
Configuration menu - View commit details
-
Copy full SHA for 1a4618a - Browse repository at this point
Copy the full SHA 1a4618aView commit details -
manifest: fix repository collection in the os pipeline
In getPackageSetChain(), the workload repositories did not include the ExtraBaseRepos. In serialize(), when creating the rpm stage options (which collects repository GPG keys), only the base repos were used, which is why we previously had to merge repositories. Instead of merging repositories in the calling function in distro, we should keep them separated so that we can easily distinguish which repositories are only meant for the blueprint or workload when we need to.
Configuration menu - View commit details
-
Copy full SHA for 30860c5 - Browse repository at this point
Copy the full SHA 30860c5View commit details -
distro: move the checkOptions() call to the top of Manifest()
There's no need to do anything if the options fail to validate, so do that first.
Configuration menu - View commit details
-
Copy full SHA for 2a55e34 - Browse repository at this point
Copy the full SHA 2a55e34View commit details -
distro: remove usage of ImageType.PackageSets() from tests
Use the new manifest generation procedure in the distro tests. Use assert instead of require in TestImageTypePipelineNames to continue running the rest of the subtests after a failure. Some initialisations (image options and blueprint customizations) had to be adjusted to work with the ImageType.Manifest() function. Some helper functions in distro_test_common are no longer necessary and have been removed. The TestPipelineRepositories and TestImageTypePipelineNames tests must be (partially) skipped for image types which specify a workload (currently only azure-eap7-rhui), because they ignore payload repositories.
Configuration menu - View commit details
-
Copy full SHA for b8b8d5b - Browse repository at this point
Copy the full SHA b8b8d5bView commit details -
distro/fedora: remove test for empty ostree ref
The ImageType.PackageSets() function is going away and instead we will rely on the ImageType.Manifest() function to both prepare the manifest and return the package sets. The Manifest() function should never be called without an ostree ref for ostree type images.
Configuration menu - View commit details
-
Copy full SHA for 92d9c7e - Browse repository at this point
Copy the full SHA 92d9c7eView commit details -
manifest: collect container and ostree source specs
Much like the GetPackageSetChains() manifest method, these two new methods collect the container and ostree source specifications from the pipelines that support them. Currently, only one pipeline per manifest contains references to containers or ostree commits, but we collect them in a map, keyed by the pipeline name, both for consistency with the package sets and for any potential future changes that may require differentiating which pipeline a content source belongs to.
Configuration menu - View commit details
-
Copy full SHA for a8afe6f - Browse repository at this point
Copy the full SHA a8afe6fView commit details -
manifest: use container SourceSpec instead of Spec
When creating a Manifest object, collect container SourceSpecs instead of resolved Specs. This is the same way we handle packages: The blueprint option is converted to source specs and attached to the Manifest object during creation. Later, the SourceSpecs will be resolved to full container Specs and used during serialization.
Configuration menu - View commit details
-
Copy full SHA for 052082c - Browse repository at this point
Copy the full SHA 052082cView commit details -
manifest: container specs added during serialization
Add a second argument, map[string][]container.Spec, during serialization, which serves the same purpose as the depsolved package sets.
Configuration menu - View commit details
-
Copy full SHA for 19a5ee3 - Browse repository at this point
Copy the full SHA 19a5ee3View commit details -
gen-manifests: resolve containers after Manifest()
Demonstrate the new workflow for resolving containers. 1. First call Manifest(). 2. Get container SourceSpecs from manifest struct. 3. Resolve them. 4. Serialize() with resolved container specs. The changes in the test manifests are just the information about the container sources (was a slice but is now a map) and the actual manifest object isn't affected. The TestDistro_Manifest test in distro_test_common is adapted accordingly as well.
Configuration menu - View commit details
-
Copy full SHA for e81aca0 - Browse repository at this point
Copy the full SHA e81aca0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 81b30dd - Browse repository at this point
Copy the full SHA 81b30ddView commit details -
weldr: resolve containers after Manifest()
Use the new workflow for generating the manifest before resolving containers. The resolver function is adjusted to handle a map of container slices, but with all current use cases, the map should only ever have one key for the payload (os) pipeline.
Configuration menu - View commit details
-
Copy full SHA for 3b4d51e - Browse repository at this point
Copy the full SHA 3b4d51eView commit details -
distro: pass containers to Serialize() in test
This has no effect since we don't use any containers in the test.
Configuration menu - View commit details
-
Copy full SHA for 7176ec8 - Browse repository at this point
Copy the full SHA 7176ec8View commit details -
distro: remove packageSpecSets and containers from Manifest() args
The arguments aren't used in the function anymore.
Configuration menu - View commit details
-
Copy full SHA for aba9ba0 - Browse repository at this point
Copy the full SHA aba9ba0View commit details -
cloudapi: remove redundant type from composite literal
Silence linter warning.
Configuration menu - View commit details
-
Copy full SHA for 1f04220 - Browse repository at this point
Copy the full SHA 1f04220View commit details -
cloudapi: remove usage of ImageType.PackageSets()
Still not using the new process for generating the manifest exactly. This commit only replaces the call to PackageSets() with a call to Manifest() to get the Content.PackageSets. This is essentially the same as before, when we were initialising the manifest object twice. The Manifest() function does a tiny bit more work than PackageSets(), but it's minimal and we gain the benefit of only having a single code path and, although it's run twice, it's always run in the same way.
Configuration menu - View commit details
-
Copy full SHA for e6b7df0 - Browse repository at this point
Copy the full SHA e6b7df0View commit details -
cloudapi: add container specs to Serialize()
To add the container specs to Serialize(), we need to map them to the payload (OS) pipeline. We assume the first name in the image type's PayloadPipelines() list is the OS pipeline, which is true of all image types right now but might not be necessarily in the future. This is a temporary workaround. Eventually, the mapping will be set by the image type itself when we use the container source specs attached to the Manifest object.
Configuration menu - View commit details
-
Copy full SHA for ee7a2cd - Browse repository at this point
Copy the full SHA ee7a2cdView commit details -
distro: remove PackageSets() from the ImageType interface
Drop the PackageSets() function completely since it's no longer needed.
Configuration menu - View commit details
-
Copy full SHA for f321275 - Browse repository at this point
Copy the full SHA f321275View commit details
Commits on May 31, 2023
-
cloudapi: set the default ostree ref for package selection
Before, this was done in the PackageSets() function. The reason for this is that having an ostree ref affects package selection (for example, it adds rpm-ostree). At the package selection phase, it doesn't matter what the ostree ref is; it is just used to determine if a pipeline is for an ostree-based image type and it doesn't affect non-ostree-based image types because the image functions ignore it. This is only needed in the cloudapi now because other places have switched to using the new order of operations, where the manifest is generated after the ostree commit is resolved, so it's always added when needed.
Configuration menu - View commit details
-
Copy full SHA for ce10100 - Browse repository at this point
Copy the full SHA ce10100View commit details