Task nxdoc 2245 build custom docker image

src/nxdoc/nuxeo-server/administration/configuration-templates.md

@@ -3,7 +3,7 @@ title: Configuration Templates
 description: Nuxeo applications integrate a configuration templates system to ease configuration and maintenance of configuration files.
 review:
     comment: ''
-    date: '2019-08-16'
+    date: '2021-01-21'
     status: ok
 labels:
     - templates
@@ -150,6 +150,7 @@ Nuxeo templates allow you to provide a specific configuration layer to any Nuxeo
 It is possible to enable one layer on a specific Nuxeo instance easily. In theory, all environments can share the configuration files and then update only the value of templates setting in the local `nuxeo.conf`.
 
 For instance, users can create templates for development, pre-production, and production environments; each template will include a different set of XML contributions:
+
 - **Users and Group**: To manage user and group authentication against different authentication systems.
 - **Database**: To store connection properties for a specific database environment.
 - **Specific needs**: To add any custom configuration, as you can add custom properties to the `nuxeo.conf` file.
@@ -162,21 +163,24 @@ Templates are located in the "templates" directory (`$NUXEO_HOME/templates`). To
 Here are the templates provided by default:
 
 {{! multiexcerpt name='default-configuration-templates'}}
-*   `common`: Common template used by other templates
-*   [`default`]({{page page='connecting-nuxeo-to-the-database'}}): default Nuxeo configuration template for test purpose
-*   [`https`]({{page page='http-and-https-reverse-proxy-configuration'}}): (not recommended) Template to make the server listen to port 443 (HTTPS)
-*   [`postgresql`]({{page page='postgresql'}}): PostgreSQL configuration template
-*   `postgresql-quartz-cluster`
-*   [`mssql`]({{page page='microsoft-sql-server'}}): MS SQL Server configuration template
-*   `mssql-quartz-cluster`
-*   [`mysql`]({{page page='mysql'}}): MySQL configuration template
-*   `mysql-quartz-cluster`
-*   [`mariadb`]({{page page='mariadb'}}): MariaDB configuration template (since Nuxeo FT 9.1; for older versions of Nuxeo the `mysql` template should be used)
-*   `mariadb-quartz-cluster`
-*   [`mongodb`]({{page page='mongodb'}}): MongoDB configuration template
-*   [`oracle`]({{page page='oracle'}}): Oracle configuration template
-*   `oracle-quartz-cluster`;
-*   `custom`: Sample custom templates. Of course, this template is empty by default. One should copy it outside `$NUXEO_HOME` and adapt to their needs. See related section below.
+
+- `common`: Common template used by other templates
+- `custom`: Sample custom templates. Of course, this template is empty by default. One should copy it outside `$NUXEO_HOME` and adapt to their needs. See related section below.
+- [`default`]({{page page='connecting-nuxeo-to-the-database'}}): default Nuxeo configuration template for test purpose
+- `docker`: Template to be used in the [Docker image]({{page page='docker-image'}}) to only append the server logs to the console, not to a `server.log` file. Also, see the page about [Logs]({{page page='logs'}}).
+- `docker-json`: Same as the `docker` template but allows to append the console logs as JSON.
+- [`https`]({{page page='http-and-https-reverse-proxy-configuration'}}): (not recommended) Template to make the server listen to port 443 (HTTPS)
+- [`mariadb`]({{page page='mariadb'}}): MariaDB configuration template (since Nuxeo FT 9.1; for older versions of Nuxeo the `mysql` template should be used)
+- `mariadb-quartz-cluster`
+- [`mongodb`]({{page page='mongodb'}}): MongoDB configuration template
+- [`mssql`]({{page page='microsoft-sql-server'}}): MS SQL Server configuration template
+- `mssql-quartz-cluster`
+- [`mysql`]({{page page='mysql'}}): MySQL configuration template
+- `mysql-quartz-cluster`
+- [`oracle`]({{page page='oracle'}}): Oracle configuration template
+- `oracle-quartz-cluster`;
+- [`postgresql`]({{page page='postgresql'}}): PostgreSQL configuration template
+- `postgresql-quartz-cluster`
 {{! /multiexcerpt}}
 
 {{#> callout type='tip' }}
@@ -209,11 +213,11 @@ You may want the variable to be resolved at runtime instead. In that case you ca
 
 The "custom" template folder allows you to add customization such as using multiple databases, configuring services, etc.
 
-1.  Add your own template files in `templates/custom` directory.
+1. Add your own template files in `templates/custom` directory.
     You can use either existing or new parameters in these new template files.
-2.  From the Admin tab or by manually [editing the `nuxeo.conf` file]({{page page='configuration-parameters-index-nuxeoconf'}}), set your parameters' values and set `nuxeo.templates=custom`.
+2. From the Admin tab or by manually [editing the `nuxeo.conf` file]({{page page='configuration-parameters-index-nuxeoconf'}}), set your parameters' values and set `nuxeo.templates=custom`.
     You can refer to custom templates directory with a relative path or to your own custom templates directory with an absolute path.
-3.  Edit `custom/nuxeo.defaults` and set `nuxeo.template.includes` parameter to define the list of existing templates to include (comma separated values); your custom template will be used at last.
+3. Edit `custom/nuxeo.defaults` and set `nuxeo.template.includes` parameter to define the list of existing templates to include (comma separated values); your custom template will be used at last.
     `nuxeo.defaults` files from included templates are also read.
 
 In case you need multiple customizations, create multiple directories and reference them in a dedicated `nuxeo.conf` for each server.
@@ -222,11 +226,11 @@ In case you need multiple customizations, create multiple directories and refere
 
 The following properties cannot be configured from a configuration template (in `nuxeo.defaults`) and must be defined in `nuxeo.conf`:
 
-*   `nuxeo.data.dir`
-*   `nuxeo.log.dir`
-*   `nuxeo.pid.dir`
-*   `nuxeo.tmp.dir`
-*   `nuxeo.mp.dir`
+- `nuxeo.data.dir`
+- `nuxeo.log.dir`
+- `nuxeo.pid.dir`
+- `nuxeo.tmp.dir`
+- `nuxeo.mp.dir`
 
 {{/callout}}
 
@@ -236,29 +240,29 @@ The "custom" template folder is a sample configuration template that you can use
 
 For instance, under Linux, you could setup:
 
-*   Configuration files (custom templates and nuxeo.conf) under `/etc/`
-
-    <pre>/etc/nuxeo/
-    ├── nuxeo.conf
-    └── some-custom-template</pre>
-
-*   Data under `/var/lib/`.
-    It is common to also place the server itself under `/var/lib/`. Other common locations for the server are `/opt/nuxeo/`, `~nuxeo/nuxeo-cap-x.y-tomcat/`...
-
-    <pre>/var/lib/nuxeo/
-    ├── data
-    └── server (NUXEO_HOME)
-    &nbsp;&nbsp;&nbsp; ├── conf
-    &nbsp;&nbsp;&nbsp; ├── lib
-    &nbsp;&nbsp;&nbsp; ├── nxserver
-    &nbsp;&nbsp;&nbsp; ├── packages
-    &nbsp;&nbsp;&nbsp; ├── templates
-    &nbsp;&nbsp;&nbsp; ├── webapps
-    &nbsp;&nbsp;&nbsp; └── work</pre>
-
-*   Log files under&nbsp;`/var/log/`
-*   PID files under&nbsp;`/var/run/`
-*   Temporary files under&nbsp;`/tmp/`
+- Configuration files (custom templates and nuxeo.conf) under `/etc/`
+
+  <pre>/etc/nuxeo/
+  ├── nuxeo.conf
+  └── some-custom-template</pre>
+
+- Data under `/var/lib/`.
+  It is common to also place the server itself under `/var/lib/`. Other common locations for the server are `/opt/nuxeo/`, `~nuxeo/nuxeo-cap-x.y-tomcat/`...
+
+  <pre>/var/lib/nuxeo/
+  ├── data
+  └── server (NUXEO_HOME)
+  &nbsp;&nbsp;&nbsp; ├── conf
+  &nbsp;&nbsp;&nbsp; ├── lib
+  &nbsp;&nbsp;&nbsp; ├── nxserver
+  &nbsp;&nbsp;&nbsp; ├── packages
+  &nbsp;&nbsp;&nbsp; ├── templates
+  &nbsp;&nbsp;&nbsp; ├── webapps
+  &nbsp;&nbsp;&nbsp; └── work</pre>
+
+- Log files under&nbsp;`/var/log/`
+- PID files under&nbsp;`/var/run/`
+- Temporary files under&nbsp;`/tmp/`
 
 * * *
 

src/nxdoc/nuxeo-server/installation/docker-image.md

@@ -1,7 +1,7 @@
 ---
 title: Docker Image
 review:
-  date: '2021-01-11'
+  date: '2021-01-26'
   status: ok
 labels:
   - multiexcerpt
@@ -11,6 +11,10 @@ description: Discover how to install the Nuxeo Platform with the Docker image.
 tree_item_index: 250
 ---
 
+## Preamble
+
+This page explains how to install the Nuxeo server with the base Nuxeo Docker image. Yet, to build an application from Nuxeo, most of the time, you will need to customize this image and build your own. For this purpose, we strongly recommend to use an immutable image by [building a custom Docker image]({{page page='build-a-custom-docker-image'}}) from the Nuxeo one.
+
 ## Requirements
 
 The only requirement to run the Nuxeo Docker image is [Docker](https://docs.docker.com/get-docker/) itself.
@@ -33,6 +37,10 @@ Based on CentOS 7, it includes:
 - The environment variables required by the server, typically `NUXEO_HOME` and `NUXEO_CONF`.
 - The exposed port `8080`.
 
+{{#> callout type='warning' }}
+To use the video related features in Nuxeo, such as conversions, storyboarding and metadata extraction, you need to have [FFMpeg](https://ffmpeg.org/) installed in the image. Please read the [FFmpeg]({{page page='build-a-custom-docker-image'}}#installing-ffmpeg) section to understand why it is not included in the Nuxeo image and how to build an image including it.
+{{/callout}}
+
 ## Running the Image
 
 Currently, the image is hosted in our private [Docker registry](https://packages.nuxeo.com/#browse/search/docker=attributes.docker.imageName%3Dnuxeo%2Fnuxeo%20AND%20attributes.docker.imageTag%3D2021*%20AND%20repository_name%3Ddocker-private).
@@ -61,7 +69,7 @@ For a production setup and general best practices, please read about [Mounting D
 
 ## Configuring the Image at Runtime
 
-Though we encourage to have immutable images configured at build time, in some cases it makes sense to configure a container at runtime. This typically applies to the address and credentials of each back-end store (database, Elasticsearch, S3, etc.) that are specific to a given deployment: development, staging, production, etc.
+Though we encourage to have immutable images [configured at build time]({{page page='build-a-custom-docker-image'}}), in some cases it makes sense to configure a container at runtime. This typically applies to the address and credentials of each back-end store (database, Elasticsearch, S3, etc.) that are specific to a given deployment: development, staging, production, etc.
 
 ### Configuration Properties
 
@@ -131,6 +139,43 @@ docker run --name nuxeo \
   docker-private.packages.nuxeo.com/nuxeo/nuxeo:2021
 ```
 
+#### NUXEO_DEV
+
+Setting `NUXEO_DEV=true` allows to run the Nuxeo image in development mode, meaning:
+
+1. Enable the following configuration properties, see the [Configuration Parameters Index (nuxeo.conf)]({{page page='configuration-parameters-index-nuxeoconf'}}) page for more details:
+
+  ```properties
+  org.nuxeo.dev=true
+  org.nuxeo.rest.stack.enable=true
+  # for hot reload
+  nuxeo.server.sdk=true
+  nuxeo.server.sdkInstallReloadTimer=true
+  ```
+
+1. Override the default command executed when running a container, allowing to execute some `nuxeoctl` commands inside the container without killing it. This way, you can for instance:
+
+    - Copy some JAR files to the `nxserver/bundles` directory of the server and restart it with `nuxeoctl restart`:
+
+    ```shell
+    docker run --name nuxeo \
+      -p 8080:8080 \
+      -e NUXEO_DEV=true \
+      docker-private.packages.nuxeo.com/nuxeo/nuxeo:2021
+
+    docker ps
+    CONTAINER ID   IMAGE
+    0eee2751d09d   docker-private.packages.nuxeo.com/nuxeo/nuxeo:2021
+
+    docker exec 0eee2751d09d nuxeoctl restart
+    ```
+
+    - Register your instance with `nuxeoctl register`.
+    - Use hot reload with the [Nuxeo Dev Tools Extension]({{page page='nuxeo-dev-tools-extension'}}).
+    - Use hot reload with [Nuxeo CLI]({{page page='nuxeo-cli'}}).
+
+1. Activate debugging for the `nuxeoctl mp-install` command run when installing Nuxeo packages with [`NUXEO_PACKAGES`](#nuxeo_packages).
+
 ### Shell Scripts
 
 To run some shell scripts when starting a container from a Nuxeo image, you can add

src/nxdoc/nuxeo-server/installation/docker-image/build-a-custom-docker-image.md

@@ -0,0 +1,98 @@
+---
+title: Build a Custom Docker Image
+review:
+  date: '2021-01-28'
+  status: ok
+labels:
+  - multiexcerpt
+  - multiexcerpt-include
+toc: true
+description: Discover how to build a custom Docker image.
+tree_item_index: 100
+---
+
+To build an application from Nuxeo, we strongly recommend to customize the Nuxeo Docker Image by building a Docker image from the Nuxeo one. Let's see how you can do this.
+
+## Build a Custom Docker Image From the Nuxeo One
+
+You can simply write a [Dockerfile](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) using the Nuxeo image as [parent image](https://docs.docker.com/glossary/#parent_image).
+A good practice is to use a [build argument](https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact) before the `FROM` instruction to easily set the version of the Nuxeo parent image, as in the example below. Here, the default value is the `2021` moving tag:
+
+```Dockerfile
+ARG NUXEO_VERSION=2021
+
+FROM docker-private.packages.nuxeo.com/nuxeo/nuxeo:${NUXEO_VERSION}
+
+# Execute some commands to add layers on top of the parent image
+```
+
+Then, the custom image can be built by running the following command in the directory of the `Dockerfile`. In this case, we choose to rely on the 2021.1 version of Nuxeo.
+
+```shell
+docker build -t mycompany/myapplication:mytag --build-arg NUXEO_VERSION=2021.1 .
+```
+
+To upgrade the custom image to a newer version of Nuxeo, for instance from 2021.1 to 2021.2, you can just rebuild the custom image by updating the `NUXEO_VERSION` build argument:
+
+```shell
+docker build -t mycompany/myapplication:mytag --build-arg NUXEO_VERSION=2021.2 .
+```
+
+Below, you can find some examples of customization that can be done in such a custom Docker image.
+
+## Installing Nuxeo Packages
+
+We provide a utility script to install remote Nuxeo packages from [Nuxeo Connect](https://connect.nuxeo.com/) and local Nuxeo packages when building an image from the Nuxeo image:
+
+For instance, you can use this script in the following `Dockerfile`:
+
+```Dockerfile
+FROM docker-private.packages.nuxeo.com/nuxeo/nuxeo:2021
+
+ARG CLID
+ARG CONNECT_URL
+
+COPY --chown=900:0 path/to/local-package-nodeps-*.zip $NUXEO_HOME/local-packages/local-package-nodeps.zip
+COPY --chown=900:0 path/to/local-package-*.zip $NUXEO_HOME/local-packages/local-package.zip
+
+# Install a local package without its dependencies (`mp-install --nodeps`)
+RUN /install-packages.sh --offline $NUXEO_HOME/local-packages/local-package-nodeps.zip
+# Install remote packages and a local package with its dependencies
+RUN /install-packages.sh --clid ${CLID} --connect-url ${CONNECT_URL} nuxeo-web-ui nuxeo-drive $NUXEO_HOME/local-packages/local-package.zip
+RUN rm -rf $NUXEO_HOME/local-packages
+```
+
+## Adding Configuration Properties
+
+As described in this [section]({{page page='configuration-parameters-index-nuxeoconf'}}#docker-image) about the `nuxeo.conf` file, the files from the `/etc/nuxeo/conf.d` directory of the Nuxeo image have their properties appended to the final `nuxeo.conf` file used by the server.
+
+If you want to embed some additional configuration properties in your custom image, you can simply copy some properties files to the `/etc/nuxeo/conf.d` directory when building the custom image.
+
+For instance:
+
+```Dockerfile
+FROM docker-private.packages.nuxeo.com/nuxeo/nuxeo:2021
+
+COPY /path/to/my-configuration.properties /etc/nuxeo/conf.d/my-configuration.properties
+```
+
+## Installing FFmpeg
+
+As it contains some non-free codecs, FFmpeg isn't part of the Nuxeo image. However, you can build a custom Docker image, based on the Nuxeo one, including the `ffmpeg` package provided by [RPM Fusion](https://rpmfusion.org/), see the `Dockerfile` sample  below. The resulting `ffmpeg` binary embeds all the codecs required for Nuxeo video conversions.
+
+```Dockerfile
+FROM docker-private.packages.nuxeo.com/nuxeo/nuxeo:2021
+
+# we need to be root to run yum commands
+USER 0
+# install RPM Fusion free repository
+RUN yum -y localinstall --nogpgcheck https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-7.noarch.rpm
+# install ffmpeg package
+RUN yum -y install ffmpeg
+# set back original user
+USER 900
+```
+
+{{#> callout type='tip' }}
+You can use the above method to install any software in the custom image.
+{{/callout}}