spel

STIG-Partitioned Enterprise Linux (spel) is a project that helps create and publish Enterprise Linux images that are partitioned according to the DISA STIG. The resulting images also use LVM to simplify volume management. The images are configured with help from the scripts and packages in the AMIgen7, AMIgen8, and AMIgen9 projects¹.

Notes on Lifecycle:

1. Images are released on a monthly cadence. This cadence ensures that, if a user launches a brand new instance from the most-recently published AMI, that there will be less than a month's worth of system-patches to apply as part of the system-owner's system-provisioning processes.
2. "Free" Enterprise Linux distributions are configured to use the public repositories offered by the distribution-owner. If running EC2s inside of a VPC with no access to the internet at large, it will not be possible to install additional RPMs or patch systems without the use of either a proxy or standing up a private yum mirror
3. Red Hat images are configured to use a given cloud service provider's (CSP) Red Hat Update Infrastructure (a.k.a., "RHUI") repositories. These repositories are managed by Red Hat engineers and provide local RPM update-service within each CSP-partner's networks. Unlike RPM-access via RHN or Satellite, RHUI access is tied to and paid for via your CSP's billing-mechanisms. RHUI access also entitles cloud-VMs' owners to limited operating system support through the respective CSP's support channels.
4. AWS Specific notes:

   • Access to the RHUI repositories is gated, in part, by an attribute attached to EC2s. This attribute is inherited from their corresponding AMIs. To view this attribute external to the EC2, execute:

     aws ec2 describe-instances --query 'Reservations[].Instances[].UsageOperation' --instance-ids
     

     This should return a value of RunInstances:0010. If the value is just RunInstances the necessary attribute is missing from the EC2.

     The attribute may also be viewed internal to the EC2 by executing:

     curl http://169.254.169.254/latest/dynamic/instance-identity/document | \
     grep "billingProducts"
     

     This should return a value of "billingProducts" : [ "bp-6fa54006" ]. If not, the necessary attribute is missing from the EC2.

     In either case, lack of the requisite attribute will mean that attempts to install or update RPMs from RHUI will fail.

   • If patch-updates should come from RHN, Satellite or other private repository, do not use the AMIs published by the maintainers of this project. Because the previously-mentioned EC2-attribute is attached to such AMIs, you will be billed for the RHUI access even if you never use it. Feel free to use this project's code to generate your own, unencumbered AMIs.

   • Further information about AWS polices for Red Hat EC2s may be found in AWS's RHEL FAQ

Why spel

VMs' root filesystems are generally not live-repartitionable once launced from their images. As a result, if a STIG-scan is performed against most of the community-published images for Red Hat and related distros (CentOS/CentOS Stream, Oracle Linux, Rocky, Alma or Liberty), those scans will note failures for each of the various "${DIRECTORY} is on its own filesystem" tests. The images produced through this project are designed to ensure that these particular scan-failures do not occur.

Aside from addressing the previously-noted partitioning findings, spel applies only those STIG-related hardenings that need to be in place "from birth" (i.e., when a system is first created from KickStart, VM-template, Amazon Machine Image, etc.). This includes things like:

• Activation of SELinux
  • Application of SELinux user-confinement to the default-user²
  • Application of SELinux role-transition rules for the default-user
• Activation of FIPS mode
• Support for BIOS- and/or EFI-boot modes (the latter being a requisite for use of SecureBoot)

The spel-produced images are expected to act as a better starting-point in a larger hardening process.

If your organization does not already have an automated hardening process, please see our tool, Watchmaker. This tool is meant to help spel-users (and users of other Enterprise Linux images) by performing launch-time hardening activities.

We have a FAQ now!

We've added an FAQ to the project. Hopefully, your questions are answered there. If they aren't, please feel free to submit an issue requesting an appropriate FAQ entry.

Current Published Images

SPEL AMIs are published monthly. The AMI table below contains links to the AWS Console that search by AMI Name and sort the result by creation date. The most recent AMI of each build will be at the top when viewed in the AWS Console.

RPM Manifests for published images are available in the manifests directory.

Please note: the RPM-manifests published to this directory are generated for the AWS (CONUS) commercial regions. Due to potential deltas between the repositories used for the commercial and govcloud regions, there may also exist deltas between what is found in the manifests in this project and the version-numbers found in the GovCloud region AMIs.

AWS Region	Builder Name / Link
us-east-1	spel-minimal-rhel-7-hvm
	spel-minimal-centos-7-hvm
	spel-minimal-rhel-8-hvm
	spel-minimal-ol-8-hvm
	spel-minimal-centos-8stream-hvm
us-east-2	spel-minimal-rhel-7-hvm
	spel-minimal-centos-7-hvm
	spel-minimal-rhel-8-hvm
	spel-minimal-ol-8-hvm
	spel-minimal-centos-8stream-hvm
us-west-1	spel-minimal-rhel-7-hvm
	spel-minimal-centos-7-hvm
	spel-minimal-rhel-8-hvm
	spel-minimal-ol-8-hvm
	spel-minimal-centos-8stream-hvm
us-west-2	spel-minimal-rhel-7-hvm
	spel-minimal-centos-7-hvm
	spel-minimal-rhel-8-hvm
	spel-minimal-ol-8-hvm
	spel-minimal-centos-8stream-hvm
us-gov-west-1	spel-minimal-rhel-7-hvm
	spel-minimal-centos-7-hvm
	spel-minimal-rhel-8-hvm
	spel-minimal-ol-8-hvm
	spel-minimal-centos-8stream-hvm
us-gov-east-1	spel-minimal-rhel-7-hvm
	spel-minimal-centos-7-hvm
	spel-minimal-rhel-8-hvm
	spel-minimal-ol-8-hvm
	spel-minimal-centos-8stream-hvm

Vagrant Cloud Name	Vagrant Provider
plus3it/spel-minimal-centos-7	virtualbox

Official AWS Owner Account IDs for Images

The following table lists the official owner accounts for the images.

AWS Partition	Account ID	Effective Release
aws	174003430611	2023.08.1 and later
aws-us-gov	216406534498	2023.08.1 and later

Deprecated AWS Owner Account IDs

The following table lists AWS account IDs previously used to host SPEL images. These accounts are now closed, and the associated images are no longer available.

AWS Partition	Account ID	Effective Release
aws	701759196663	2023.07.1 and earlier
aws-us-gov	039368651566	2023.07.1 and earlier

Deprecated CentOS 8 Images

With the move from CentOS 8 to CentOS Stream 8, the CentOS 8 images are deprecated. While they remain public for the moment, they are no longer updated and the CentOS org may remove the yum repos at their discretion.

AWS Region	Builder Name / Link
us-east-1	spel-minimal-centos-8-hvm
us-east-2	spel-minimal-centos-8-hvm
us-west-1	spel-minimal-centos-8-hvm
us-west-2	spel-minimal-centos-8-hvm
us-gov-west-1	spel-minimal-centos-8-hvm
us-gov-east-1	spel-minimal-centos-8-hvm

Default Username

The default username for all spel images is maintuser.

If you wish to change the default username at launch, you can do so via cloud-init with userdata³ something like the following. Change <USERNAME> to your desired value.

#cloud-config
system_info:
  default_user:
    name: <USERNAME>
    gecos: spel default user
    lock_passwd: true

Default User Security-Constraints

Due to updates to the STIGs – currently just for EL7, but it is assumed that similar changes for EL8 and later distros will be added to future STIG-releases – the default-user's account may have additional SELinux rules applied to it. These rules will typically manifest in processes that start as the default-user (i.e., processes run as the root user after privilege-escalation via the sudo subsystem) receiving permission denied errors when attempting to access "sensitive" files. These "sensitive" files are any that have the shadow_t SELinux context-label applied to them. By default, these will only include:

• /etc/security/opasswd
• /etc/shadow
• /etc/gshadow

A definitive list may be gathered by executing the command:

find / -context "*shadow_t*"`

If your workflows absolutely require the ability to access these files after a role-transition from the default-user account to root, it will be necessary to update the userData payload's cloud-config content to include a block similar to:

#cloud-config
system_info:
  default_user:
    name: <USERNAME>
    gecos: spel default user
    lock_passwd: true
    selinux_user: unconfined_u
    sudo: ["ALL=(root) NOPASSWD:ALL"]

However, doing so will result in security scan-failures when the scanning-tool tries to ensure that all locally-managed, interactive users are properly-constrained users and, where appropriate, have SELinux privilege-transition rules defined.

Prerequisites

Packer by Hashicorp is used to manage the process of building images.

1. Download and extract packer for your platform. Add it to your PATH, if you like. On Linux, watch out for other packer executables with the same name (if building from an Enterprise Linux distro, /sbin/packer may be present due to the cracklib-dicts RPM).

2. If building AMIs for Amazon Web Services, ensure your AWS credentials are configured. You do not really need the aws cli utility, but it is a convenient way to configure the credential file. You can also export the environment variables. Or, if running packer in an EC2 instance, an instance role with the requisite permissions will also work. See the packer docs for details on the necessary permissions.

   NOTE: No packer templates in this project will contain variables for AWS credentials; this is intentional, to avoid mistakes where credentials get committed to the repository. Instead, packer knows to read the credentials from the credential file or from the environment variables, or to retrieve them from the instance role. See the docs.

3. If building VirtualBox image(s), you will need to install VirtualBox and Vagrant.

4. If building VMware image(s), depending on your platform, you will need to install either VMware Fusion, VMware Workstation Pro, or VMware Player. For all platforms, you will also need Vagrant.

5. The template(s) push the Vagrant boxes for the VirtualBox and VMware images to Hashicorp Vagrant Cloud, which requires a Vagrant Cloud account.

6. If building a VHD or Image for Azure, ensure you have authorized access to ARM. The creation of destination objects and a Service Principal can either be done manually or via script. If not building in Public region, use of device login is not possible and a Service Principal is required.

Usage

NOTE: In all steps below, the examples use syntax that works on Linux. If you are running packer from a Windows system, simply use the appropriate syntax for the relative path to the packer template. Most important, for Windows, use .\ preceding the path to the template. E.g. .\spel\minimal-linux.json.

1. Clone the repository:

   git clone https://github.com/plus3it/spel && cd spel

2. Validate the template (Optional):

   packer validate spel/minimal-linux.pkr.hcl

   The project-included Packer HCL files have been pre-validated. If you encounter validation-errors with the included HCL files, it means that you're using a newer Packer version than the project has been tested against. Please open an issue to report the problem, ensuring to include the Packer version you were using when you encountered the problem.

3. Begin the build. This requires at least two variables, spel_identifier and spel_version. See the section Packer Variables for more details.

   packer build \
       -var 'spel_identifier=unique-project-id' \
       -var 'spel_version=dev001' \
       -var 'virtualbox_vagrantcloud_username=myvagrantclouduser' \
       spel/minimal-linux.pkr.hcl

   NOTE: This will build images for all the builders defined in the template. Use packer build --help to see how to restrict the build to to a subset of the builders using the -only or -except arguments.

   If building the VirtualBox or VMware images for use with Vagrant, the template is configured to host the resulting images with Hashicorp Vagrant Cloud. This requires passing the variable virtualbox_vagrantcloud_username and exporting the environment variable VAGRANT_CLOUD_TOKEN.

Minimal Linux Packer Template

The Minimal Linux template builds STIG-partitioned images with a set of packages that correspond to the "Minimal" install option in Anaconda. Further, the AWS images include a handful of additional packages that are intended to increase functionality in EC2 and make the images more comparable with Amazon Linux. Similarly, the Azure builder will attempt to install the WALinuxAgent RPM into the VM-template to make the template more integratable into Azure-based deployments.

• Template Path: spel/minimal-linux.pkr.hcl

For all inputs to the template, see spel/README.md

Minimal Linux Packer Builders

The Minimal Linux packer template includes the following builders:

Builder Name	Description
amazon-ebssurrogate.minimal-centos-8stream-hvm	amazon-ebs builder for a minimal CentOS Stream 8 HVM AMI
amazon-ebssurrogate.minimal-ol-8-hvm	amazon-ebs builder for a minimal Oracle Linux 8 HVM AMI
amazon-ebssurrogate.minimal-rhel-8-hvm	amazon-ebs builder for a minimal RHEL 8 HVM AMI
amazon-ebssurrogate.minimal-centos-7-hvm	amazon-ebs builder for a minimal CentOS 7 HVM AMI
amazon-ebssurrogate.minimal-rhel-7-hvm	amazon-ebs builder for a minimal RHEL 7 HVM AMI
azure-arm.minimal-centos-7-image	azure-arm builder for a minimal CentOS 7 Image
azure-arm.minimal-rhel-7-image	azure-arm builder for a minimal RHEL 7 Image
azure-arm.minimal-rhel-8-image	azure-arm builder for a minimal RHEL 8 Image
openstack.minimal-centos-7-image	openstack builder for a minimal CentOS 7 Image
virtualbox-iso.minimal-centos-7-image	virtualbox-iso builder for a minimal CentOS 7 Vagrant Box

Minimal Linux Packer Post-Provisioners

The Minimal Linux packer template includes the following post-provisioners:

• vagrant: The vagrant post-provisioner creates vagrant boxes from on the virtualbox and vmware images.

• vagrant-cloud: The vagrant-cloud post-provisioners upload the vagrant boxes to Hashicorp Vagrant Cloud.

Building for the AWS US GovCloud Region

To build images for the AWS US GovCloud regions, us-gov-west-1 or us-gov-east-1, it is necessary to pass several variables that are specific to the region. The AMI filters below have been tested and/or created in us-gov-west-1 to work with the spel template(s). Also, the builders should be restricted so as not to build the Vagrant images.

packer build \
    -var 'spel_identifier=unique-project-id' \
    -var 'spel_version=dev001' \
    -var 'aws_region=us-gov-west-1' \
    -var 'aws_source_ami_filter_centos7_hvm={name = "*-Recovery (No-LVM)-ACB-CentOS7-HVM-SRIOV_ENA", owners = ["216406534498"]}' \
    -var 'aws_source_ami_filter_centos8stream_hvm={name = "spel-bootstrap-centos-8stream-hvm-*.x86_64-gp*", owners = ["216406534498"]}' \
    -exclude 'virtualbox-iso.*' \
    spel/minimal-linux.pkr.hcl

Building for Microsoft Azure

A source Marketplace Image Offer or Custom Image Name and Resource Group are required from which to start the SPEL Azure build.

The resultant SPEL Image will be configured to use the Azure Linux agent, WALinuxAgent per recommended configurations. Currently, the use of cloud-init exclusively does not enable execution/installation of Azure VM Extensions. The below variables also disable FIPS mode in the resultant SPEL VHD or Image. Currently, the Azure Linux agent does not support FIPS mode when utilizing Azure VM Extensions. If no plans exist to utilize Azure VM Extensions on VMs provisioned from SPEL VHDs or Images, FIPS mode can be enabled, but the waagent configuration must also be modified accordingly.

The variables referenced in the packer builds below should be modified with appropriate parameters for your environment. Any content between and including the < and > characters should be replaced.

Login to azure using the az cli. Packer will use the session setup by the az cli.

packer build \
    -var 'spel_identifier=unique-project-id' \
    -var 'spel_version=0.0.1' \
    -var 'amigen_extra_rpms=["WALinuxAgent"]' \
    -var 'amigen_fips_disable=true' \
    -var 'amigen7_repo_names=["rhui-microsoft-azure-rhel7"]' \
    -var 'azure_image_offer=rhel-raw' \
    -var 'azure_image_publisher=RedHat' \
    -var 'azure_image_sku=7-raw' \
    -var 'azure_managed_image_resource_group_name=<resource group short name>' \
    -only 'azure-arm.minimal-rhel-7-image' \
    spel/minimal-linux.pkr.hcl

When building for RHEL 8:

• Change the -only flag to reference azure-arm.minimal-rhel-8-image
• Change the azure_image_sku to an appropriate value. When the azure-arm.minimal-rhel-8-image was being authored, the appropriate value was 8_8
• Substitute the amigen8_repo_names variable for the amigen7_repo_names and set an appropriate list of RHUI repositories to support RHEL 8

Building for OpenStack

To build images for an OpenStack environment, it is necessary to pass several variables that are specific to the environment. The CentOS 7 Generic Cloud image has been tested to work with the spel template(s). Also, the builders should be restricted so as not to build the Vagrant images.

source your_openstack_credentials_file.sh
packer build \
    -var 'spel_identifier=spel' \
    -var 'spel_version=0.0.1' \
    -var 'openstack_insecure=false' \
    -var 'openstack_flavor=your_flavor_name_for_temporary_instance' \
    -var 'openstack_floating_ip_network=your_provider_network_name' \
    -var 'openstack_networks=your_network_id_for_temporary_instance,second_network_id,etc.' \
    -var 'openstack_security_groups=your_security_group_name_for_temporary_instance,second_sg_name,etc.' \
    -var 'openstack_source_image_name=your_source_image_name' \
    -only 'openstack.*' \
    spel/minimal-linux.pkr.hcl

For expected values, see links below:

• openstack_allow_insecure (true|false)
• openstack_flavor_name (string)
• openstack_floating_ip_network_name (string)
• openstack_network_ids (comma-separated list of strings)
• openstack_security_group_names (comma-separated list of strings)
• openstack_source_image_name (string)

Testing With AMIgen

The spel automation leverages the AMIgen7 and AMIgen8 projects as a build-helpers for creation of EL7 and EL8 Amazon Machine Images (Azure VM-templates, etc.), respectively. Due to the closely-coupled nature of the two projects, it's recommended that any changes made to AMIgen7 or AMIgen8 be tested with spel prior to merging changes to either project's master branch.

To facilitate this testing, the following runtime-variables were added to spel:

• amigen7_source_branch
• amigen7_source_url
• amigen8_source_branch
• amigen8_source_url

Using these runtime-variables allows one to point spel to a fork/branch of AMIgen7 or AMIgen8 during a integration-test build. To test, update your packer invocation by adding elements like:

packer build \
    -var 'amigen7_source_url=https://github.com/<FORK_USER>/AMIgen7.git' \
    -var 'amigen7_source_branch=IssueNN' \
    ...
    minimal-linux.pkr.hcl

Similarly, these variable may be specified as environment variables by using PKR_VAR_<var_name> declarations⁴ (e.g., PKR_VAR_amigen7_source_branch). To do so, change the above example to:

export PKR_VAR_amigen7_source_branch="=https://github.com/<FORK_USER>/AMIgen7.git"
export PKR_VAR_amigen7_source_branch="IssueNN"

packer build \
    [...options elided...]
    minimal-linux.pkr.hcl

Footnotes

1. Because spel is primarily an execution-wrapper for the AMIgenN projects, the "read the source" method for determining why things have changed from one spel-release to the next may require reviewing those projects' repositories ↩

2. The default-user is a local user (i.e., managed in /etc/passwd//etc/shadow//etc/group) that is dynamically-created at initial system-boot – using either the default-information in the /etc/cloud/cloud.cfg file or as overridden in a userData payload's #cloud-config content. Typically this user's ${HOME}/.ssh/authorized_keys file is prepopulated with a provisioner's public SSH key. ↩

3. Overriding attributes of the default-user must be done within a #cloud-config directive-block. If your userData is currently bare BASH (etc.), it will be necessary to format your userData payload as mixed, multi-part MIME. ↩

4. Use of the PKR_VAR_ method is recommended for setting up CI/CD frameworks for producing AMIs and other supported VM-templates ↩