Skip to content
Permalink
Browse files
Updates to site-content build tools
* Cassandra configuration and nodetool pages are no longer committed to git. The changes makes the run.sh -i and -n options redundant and so they have been removed.
  - README. md has been updated to reflect this change
* Changed site-content Docker file
  - Installed JDK 11 for compiling trunk and cassandra-4.0 branches
  - Made 'build' user a sudo user
* Changed the documentation generation function in the docker-entrypoint.sh
  - Always makes a copy of the repository if is supplied from the local host calling the container
  - Select the version of JDK to use based on the version of Cassandra being built

 patch by Anthony Grasso; reviewed by Mick Semb Wever for CASSANDRA-16763
  • Loading branch information
ossarga authored and michaelsembwever committed Dec 20, 2021
1 parent f638579 commit 60dc37bfe004ca9b5e8b31ff794e987a5ced1774
Showing 4 changed files with 154 additions and 121 deletions.
@@ -83,11 +83,19 @@ $ ./run.sh website container -a BUILD_USER_ARG:$(whoami) -a UID_ARG:$(id -u) -a

If you need to customise the container user as noted above, you must do this before you build the website or run any other website command.

## Build the Website when Developing
## Building the Website with Versioned Documentation

To build the website with versioned documentation run the following command from within the `./cassandra-website` directory.

```bash
$ ./run website build -g
```

# Build the Website when Developing

The website tooling is very flexible and allows for a wide range of development scenarios.

### Build the website from a different branch
## Build the website from a different branch

You can tell the website builder to use a different branch to the one you are on. This can be done using the following command.

@@ -97,7 +105,7 @@ $ ./run.sh website build -b cassandra-website:my_branch

This will build the website content using your local copy of the cassandra-website, and the branch named `my_branch`.

### Build the website using a local clone of the repository
## Build the website using a local clone of the repository

You can tell the website builder to use a different clone or fork of the repository.

@@ -109,7 +117,7 @@ $ ./run.sh website build -u cassandra-website:/local/path/to/another/clone/of/ca

This will build the website using the contents of the local repository located in */local/path/to/another/clone/of/cassandra-website*

### Build the website using a remote clone of the repository
## Build the website using a remote clone of the repository

To build using a remote copy of the cassandra-website run the following command.

@@ -145,7 +153,7 @@ All options that are available in the `build` command can be used by the `previe

The cassandra-website tooling can also be used to perform a number of other operations:

* Generate the Apache Cassandra documentation.
* Generate the Apache Cassandra documentation alone.
* Update the website styling and behaviour.

## Generating the Cassandra Documentation
@@ -184,32 +192,32 @@ In the above command, multiple branches separated by a comma (`,`) can be specif

The website tooling can be used to run a complete end-to-end generation of the HTML website and versioned documentation. That is, it can first generate the AsciiDoc files for each Cassandra version, and then launch `antora` to generate the HTML website and versioned documentation for publication.

### Generate the website and documentation using pre-generated Cassandra AsciiDoc in local repositories
### Generate the website and documentation from a local repositories

If you have already generated the Cassandra AsciiDoc (`.adoc`) files and committed them to your repository, you can skip the Cassandra AsciiDoc generation process.
You can use a local copy of the Cassandra repository as the source for the documentation.

To build the website using a local clone of the Cassandra repository that contains the generated AsciiDoc files, and the Cassandra Website run the following command.
To build the website using a local clone of the Cassandra repository run the following command.

```bash
$ ./run.sh \
website \
build \
-i \
-g \
-u cassandra:/local/path/to/cassandra/repository \
-u cassandra-website:/local/path/to/cassandra-website/repository
```

In the above command, the `-i` option is used to tell the tooling to include the Cassandra repository when `antora` is generating the HTML. This ensures the versioned documentation HTML is generated along with the website HTML. In this case, exiting AsciiDoc files in the Cassandra repository are used to generate the versioned documentation HTML. That is, no additional operations are run to pre-generate the Cassandra AsciiDoc files.
In the above command, the `-g` option is used to tell the tooling to generate the non-committed AsciiDoc files and include the Cassandra repository as a source when `antora` is generating the HTML. This ensures the complete versioned documentation HTML is generated along with the website HTML.

### Generate the website and documentation using pre-generated Cassandra AsciiDoc in remote repositories
### Generate the website and documentation from a remote repositories

To build using a remote copy of the Cassandra repository that contains the generated AsciiDoc files, and the Cassandra Website run the following command.

```bash
$ ./run.sh \
website \
build \
-i \
-g \
-u cassandra:https://github.com/my_orgranisation/cassandra.git \
-u cassandra-website:https://github.com/my_orgranisation/cassandra-website.git
```
@@ -220,7 +228,7 @@ You can have a combination of local and remote repository paths. For example, yo
$ ./run.sh \
website \
build \
-i \
-g \
-u cassandra:https://github.com/my_orgranisation/cassandra.git \
-u cassandra-website:/local/path/to/cassandra-website/repository
```
78 run.sh
@@ -4,6 +4,15 @@ debug() {
>&2 echo "[DEBUG] $*"
}

error() {
echo
echo "ERROR: $1."
echo
echo "For help run:"
echo "$ ./run.sh help"
exit 1
}

usage() {
cat <<EOF
This script builds the Cassandra website UI components, generates the Cassandra versioned documentation, and renders the website.
@@ -132,29 +141,14 @@ Options
'build' command, and is ignored in all other cases. Possible arguments for BUILD_ARG
are 'BUILD_USER_ARG', 'UID_ARG', 'GID_ARG', 'NODE_VERSION_ARG', 'ENTR_VERSION_ARG'.
The build argument and value are split by a colon ':'. For example:
-v BUILD_USER_ARG:foobar
-a BUILD_USER_ARG:foobar
-g Enable generation of versioned AsciiDoc (.adoc) files from the Cassandra source
repository and include them when generating the website HTML. Use this option when you
want to generate a new version of the Cassandra AsciiDocs before generating the
website and documentation HTML. This option can be used with only the website 'build',
and 'preview' commands. For all other commands this option is ignored.
-i Enable the inclusion of the versioned AsciiDoc (.adoc) files from the Cassandra source
repository when generating the website HTML. This option is unnecessary when the '-g'
option is used. Use this option when the AsciiDoc files are already committed to the
Cassandra source repository and you want them to be used for the versioned docs in the
generated HTML website. This option can be used with only the website 'build', and
'preview' commands. For all other commands this option is ignored.
-n Disable committing the generated docs to the Cassandra repository. This option can be
used when only a single branch is specified as the Cassandra repository source. If
using multiple branches, the AsciiDoc files generated for each branch need to be
committed to their respective branch. This is so Antora can generate the website HTML
documentation for each branch. Use this option when you use your local checkout of the
Cassandra repository and want to commit any newly generated AsciiDoc files to the
repository yourself.
-p Preserve containers after docker exists. By default this script will force docker to
remove the container once it stops. Use this option to persist the container after it
has finished running.
@@ -170,11 +164,11 @@ Options
-d Dry run. Only display the docker command that will be executed but never execute it.
EOF
exit 0
exit $1
}

parse_website_command_options() {
while getopts "e:c:b:t:u:z:a:ginpdh" opt_flag
while getopts "e:c:b:t:u:z:a:gpd" opt_flag
do
case $opt_flag in
e)
@@ -200,25 +194,15 @@ parse_website_command_options() {
;;
g)
command_generate_docs="generate-docs"
include_version_docs_when_generating_website="enabled"
;;
i)
include_version_docs_when_generating_website="enabled"
;;
n)
commit_generated_version_docs_to_repository="disabled"
;;
p)
persist_container_after_run="enabled"
;;
d)
dry_run="enabled"
;;
h)
usage
;;
*)
usage
usage 1
;;
esac
done
@@ -261,8 +245,7 @@ parse_file_uri() {
# Strip out the "localhost" so we are just left with the path
file_uri=${file_uri//localhost}
else
echo "ERROR: Path ${location_source} is for a file or directory on a remote server. Please specify a localhost path or HTTP URL"
exit 1
error "Path ${location_source} is for a file or directory on a remote server. Please specify a localhost path or HTTP URL"
fi
fi

@@ -314,9 +297,9 @@ get_source_location_information() {

if [ -n "${relative_path}" ]
then
pushd "${relative_path}" > /dev/null || { echo "ERROR: Failed to change directory to '${relative_path}'."; exit 1; }
pushd "${relative_path}" > /dev/null || { error "Failed to change directory to '${relative_path}'."; }
location_source=$(pwd)
popd > /dev/null || { echo "ERROR: Failed to change back to working directory after changing to '${relative_path}'."; exit 1; }
popd > /dev/null || { error "Failed to change back to working directory after changing to '${relative_path}'."; }
fi

if [ -n "${file_name}" ]
@@ -521,8 +504,6 @@ EOF
env_args+=("-e ANTORA_CONTENT_SOURCES_CASSANDRA_WEBSITE_URL=${container_build_dir}/cassandra-website")
fi

env_args+=("-e INCLUDE_VERSION_DOCS_WHEN_GENERATING_WEBSITE=${include_version_docs_when_generating_website}")

# CASSANDRA-16913:
# Include the local site-ui/build/ui-bundle.zip automatically, unless an alternative path has been defined via the
# '-z' option. If we are on a custom branch, we would expect the local site-ui/build/ui-bundle.zip to be used from
@@ -538,9 +519,7 @@ EOF
env_args+=("-e ANTORA_UI_BUNDLE_URL=${url_source_value}")
fi

env_args+=("-e COMMIT_GENERATED_VERSION_DOCS_TO_REPOSITORY=${commit_generated_version_docs_to_repository}")

exec_docker_run_command "${port_map_option} ${vol_args[*]} ${env_file_arg} ${env_args[*]} ${container_tag} ${command_generate_docs} ${container_command}"
exec_docker_run_command --name "website_content" "${port_map_option} ${vol_args[*]} ${env_file_arg} ${env_args[*]} ${container_tag} ${command_generate_docs} ${container_command}"
}

run_website_command() {
@@ -567,6 +546,10 @@ run_website_command() {
preview)
run_docker_website_command "preview"
;;

*)
error "Unrecognised website command - ${command}"
;;
esac
}

@@ -591,9 +574,9 @@ run_docker_website_ui_command() {

if [ "${container_command}" = "help" ]
then
exec_docker_run_command -v "${volume_map_option}" "${container_tag}"
exec_docker_run_command --name "website_ui" -v "${volume_map_option}" "${container_tag}"
else
exec_docker_run_command "${port_map_option}" -v "${volume_map_option}" "${container_tag}" "${container_command}"
exec_docker_run_command --name "website_ui" "${port_map_option}" -v "${volume_map_option}" "${container_tag}" "${container_command}"
fi
}

@@ -630,8 +613,6 @@ repository_tags=()
repository_url=()
ui_bundle_zip_url=""
command_generate_docs=""
commit_generated_version_docs_to_repository="enabled"
include_version_docs_when_generating_website="disabled"
persist_container_after_run="disabled"
dry_run="disabled"

@@ -640,15 +621,15 @@ arg_command=""

if [ "$#" -eq 0 ]
then
usage
error "Missing component argument"
fi

arg_component="$1"
shift 1

if [ "$#" -eq 0 ]
if [ "$#" -eq 0 ] && [ "${arg_component}" != "help" ]
then
usage
error "Missing command argument"
fi

arg_command="$1"
@@ -660,6 +641,7 @@ docker_output=""

if ! docker_output=$(docker version)
then
echo
echo -n "ERROR: "
tail -n 1 <<<"${docker_output}"
else
@@ -676,7 +658,11 @@ case "${arg_component}" in
run_website_ui_command "${arg_command}"
;;

help)
usage 0
;;

*)
usage
error "Unrecognised component - ${arg_component}"
;;
esac
@@ -17,32 +17,27 @@ ARG GID_ARG=1000
ARG NODE_VERSION_ARG="v12.16.2"
ARG ENTR_VERSION_ARG="4.6"

ENV BUILD_USER=${BUILD_USER_ARG}

RUN echo "Building with arguments:" \
&& echo " - BUILD_USER_ARG=${BUILD_USER_ARG}" \
&& echo " - UID_ARG=${UID_ARG}" \
&& echo " - GID_ARG=${GID_ARG}" \
&& echo " - NODE_VERSION_ARG=${NODE_VERSION_ARG}" \
&& echo " - ENTR_VERSION_ARG=${ENTR_VERSION_ARG}"

RUN echo "Setting up user '${BUILD_USER}'"
RUN groupadd --gid ${GID_ARG} --non-unique ${BUILD_USER}
RUN useradd --create-home --shell /bin/bash \
--uid ${UID_ARG} --gid ${GID_ARG} --non-unique ${BUILD_USER}

# INSTALL wget, python3, java11, and other tools required to build the docs
RUN apt-get update && \
apt-get install -y \
wget \
gpg \
RUN apt update && \
apt install -y \
openjdk-8-jdk \
openjdk-11-jdk \
python3 \
python3-pip \
openjdk-11-jdk \
git \
make \
ant \
ant-optional
ant-optional \
make \
git \
gpg \
wget \
sudo

RUN pip3 install jinja2 requests

@@ -57,9 +52,15 @@ RUN npm i -g @antora/cli@2.3 @antora/site-generator-default@2.3 @djencks/asciido
RUN npm i -g antora-lunr antora-site-generator-lunr
RUN npm i -g live-server

# Create the build user and make it part of the password-less sudo group
ENV BUILD_USER=${BUILD_USER_ARG}
RUN groupadd --gid ${GID_ARG} --non-unique ${BUILD_USER} && \
useradd --create-home --shell /bin/bash --uid ${UID_ARG} --gid ${GID_ARG} --non-unique ${BUILD_USER} && \
usermod -aG sudo ${BUILD_USER} && \
sed -i 's/\(^%sudo\tALL=(ALL:ALL)\).*/\1 NOPASSWD:ALL/g' /etc/sudoers

# Setup directories for building the docs
# Give the build user rw access to everything in the build directory,
# neccessary for the ASF 'website'.
# Give the build user rw access to everything in the build directory neccessary for the ASF 'website'.
ENV BUILD_DIR="/home/${BUILD_USER}"
ENV ENTR_PACKAGE="${ENTR_VERSION_ARG}.tar.gz"
WORKDIR ${BUILD_DIR}
@@ -102,9 +103,6 @@ ENV ANTORA_UI_BUNDLE_URL="https://github.com/apache/cassandra-website/raw/trunk/

ENV CASSANDRA_DOWNLOADS_URL="https://downloads.apache.org/cassandra/"

ENV INCLUDE_VERSION_DOCS_WHEN_GENERATING_WEBSITE="disabled"
ENV COMMIT_GENERATED_VERSION_DOCS_TO_REPOSITORY="enabled"

EXPOSE 5151/tcp

# Run as build user from here

0 comments on commit 60dc37b

Please sign in to comment.