Update documentation (#4700) [ci skip]

Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Co-authored-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

docs/cli.md

@@ -50,6 +50,9 @@ Available options:
 `-syslog`
 : Send logs to syslog server (e.g. localhost:514).
 
+`-trace`
+: Enable trace level logging for the specified packages. Multiple packages can be provided separating them with a comma, e.g. `-trace nextflow,io.seqera`.
+
 `-v, -version`
 : Print the program version.
 
@@ -121,7 +124,7 @@ The `-c` option is used to append a new configuration to the default configurati
 
 ### Docker driven execution
 
-:::{warning} *Experimental: not recommended for production environments.*
+:::{deprecated} 23.09.0-edge
 :::
 
 Launch Nextflow via Docker.
@@ -1129,7 +1132,7 @@ The `run` command is used to execute a local pipeline script or remote pipeline
 : Enable/disable ANSI console logging.
 
 `-bucket-dir`
-: Remote bucket where intermediate result files are stored.
+: Remote bucket where intermediate result files are stored. When running a hybrid workflow, `-bucket-dir` and `-work-dir` should define separate work directories for remote tasks and local tasks, respectively.
 
 `-cache`
 : Enable/disable processes caching.

docs/config.md

@@ -538,7 +538,7 @@ The following settings are available:
 
 `dag.direction`
 : :::{versionadded} 23.10.0
-:::
+  :::
 : *Only supported by the HTML and Mermaid renderers.*
 : Controls the direction of the DAG, can be `'LR'` (left-to-right) or `'TB'` (top-to-bottom) (default: `'TB'`).
 
@@ -986,7 +986,7 @@ The `k8s` scope controls the deployment and execution of workflow applications i
 The following settings are available:
 
 `k8s.autoMountHostPaths`
-: Automatically mounts host paths in the job pods. Only for development purpose when using a single node cluster (default: `false`).
+: Automatically mounts host paths into the task pods (default: `false`). Only intended for development purposes when using a single node.
 
 `k8s.computeResourceType`
 : :::{versionadded} 22.05.0-edge
@@ -996,6 +996,9 @@ The following settings are available:
 `k8s.context`
 : Defines the Kubernetes [configuration context name](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) to use.
 
+`k8s.debug.yaml`
+: When `true`, saves the pod spec for each task to `.command.yaml` in the task directory (default: `false`).
+
 `k8s.fetchNodeName`
 : :::{versionadded} 22.05.0-edge
   :::
@@ -1029,6 +1032,7 @@ The following settings are available:
 
 `k8s.pod`
 : Allows the definition of one or more pod configuration options such as environment variables, config maps, secrets, etc. It allows the same settings as the {ref}`process-pod` process directive.
+: When using the `kuberun` command, this setting also applies to the submitter pod.
 
 `k8s.projectDir`
 : Defines the path where Nextflow projects are downloaded. This must be a path in a shared K8s persistent volume (default: `<volume-claim-mount-path>/projects`).
@@ -1581,7 +1585,13 @@ There are additional variables that can be defined within a configuration file t
   :::
 
 `dumpHashes`
-: If `true`, dump task hash keys in the log file, for debugging purposes.
+: If `true`, dump task hash keys in the log file, for debugging purposes. Equivalent to the `-dump-hashes` option of the `run` command.
+
+`resume`
+: If `true`, enable the use of previously cached task executions. Equivalent to the `-resume` option of the `run` command.
+
+`workDir`
+: Defines the pipeline work directory. Equivalent to the `-work-dir` option of the `run` command.
 
 (config-profiles)=
 
@@ -1711,7 +1721,6 @@ The following environment variables control the configuration of the Nextflow ru
   :::
 : Enable to use of AWS SES native API for sending emails in place of legacy SMTP settings (default: `false`)
 
-
 `NXF_ENABLE_FS_SYNC`
 : :::{versionadded} 23.10.0
   :::
@@ -1727,8 +1736,16 @@ The following environment variables control the configuration of the Nextflow ru
   :::
 : Enable Nextflow *strict* execution mode (default: `false`)
 
+`NXF_ENABLE_VIRTUAL_THREADS`
+: :::{versionadded} 23.05.0-edge
+  :::
+: :::{versionchanged} 23.10.0
+  Enabled by default when using Java 21 or later.
+  :::
+: Enable the use of virtual threads in the Nextflow runtime (default: `false`)
+
 `NXF_EXECUTOR`
-: Defines the default process executor e.g. `sge`
+: Defines the default process executor, e.g. `sge`
 
 `NXF_FILE_ROOT`
 : :::{versionadded} 23.05.0-edge
@@ -1747,6 +1764,9 @@ The following environment variables control the configuration of the Nextflow ru
   :::
 : Allows the setting Java VM options. This is similar to `NXF_OPTS` however it's only applied the JVM running Nextflow and not to any java pre-launching commands.
 
+`NXF_LOG_FILE`
+: The filename of the Nextflow log (default: `.nextflow.log`)
+
 `NXF_OFFLINE`
 : When `true` prevents Nextflow from automatically downloading and updating remote project repositories (default: `false`).
 : :::{versionchanged} 23.09.0-edge
@@ -1805,6 +1825,9 @@ The following environment variables control the configuration of the Nextflow ru
 `NXF_TEMP`
 : Directory where temporary files are stored
 
+`NXF_TRACE`
+: Enable trace level logging for the specified packages. Equivalent to the `-trace` command-line option.
+
 `NXF_VER`
 : Defines which version of Nextflow to use.
 

docs/fusion.md

@@ -186,7 +186,7 @@ The following configuration options are available:
 
 `fusion.cacheSize`
 : :::{versionadded} 23.11.0-edge
-:::
+  :::
 : The maximum size of the local cache used by the Fusion client.
 
 `fusion.containerConfigUrl`

docs/mail.md

@@ -36,20 +36,39 @@ sendMail(mail)
 
 The following parameters can be specified:
 
-| Name          | Description                                                            |
-| ------------- | ---------------------------------------------------------------------- |
-| to {sup}`*`   | The mail target recipients.                                            |
-| cc {sup}`*`   | The mail CC recipients.                                                |
-| bcc {sup}`*`  | The mail BCC recipients.                                               |
-| from {sup}`*` | The mail sender address.                                               |
-| subject       | The mail subject.                                                      |
-| charset       | The mail content charset (default: `UTF-8`).                           |
-| text          | The mail plain text content.                                           |
-| body          | The mail body content. It can be either plain text or HTML content.    |
-| type          | The mail body mime type. If not specified it's automatically detected. |
-| attach        | Single file or a list of files to be included as mail attachments.     |
-
-`*` Multiple email addresses can be specified separating them with a comma.
+`to`
+: *Multiple email addresses can be specified separating them with a comma.*
+: The mail target recipients.
+
+`cc`
+: *Multiple email addresses can be specified separating them with a comma.*
+: The mail CC recipients.
+
+`bcc`
+: *Multiple email addresses can be specified separating them with a comma.*
+: The mail BCC recipients.
+
+`from`
+: *Multiple email addresses can be specified separating them with a comma.*
+: The mail sender address.
+
+`subject`
+: The mail subject.
+
+`charset`
+: The mail content charset (default: `UTF-8`).
+
+`text`
+: The mail plain text content.
+
+`body`
+: The mail body content. It can be either plain text or HTML content.
+
+`type`
+: The mail body mime type. If not specified it's automatically detected.
+
+`attach`
+: Single file or a list of files to be included as mail attachments.
 
 (mail-advanced)=
 
@@ -89,14 +108,19 @@ To send an email that includes text and HTML content, use both the `text` and `b
 
 When using the curly brackets syntax, the `attach` parameter can be repeated two or more times to include multiple attachments in the mail message.
 
-Moreover for each attachment it's possible to specify one or more of the following optional attributes:
+Moreover for each attachment it's possible to specify any of the following options:
+
+`contentId`
+: Defines the `Content-ID` header field for the attachment.
+
+`disposition`
+: Defines the `Content-Disposition` header field for the attachment.
+
+`fileName`
+: Defines the `filename` parameter of the `Content-Disposition` header field.
 
-| Name        | Description                                                                 |
-| ----------- | --------------------------------------------------------------------------- |
-| contentId   | Defines the `Content-ID` header field for the attachment.                   |
-| disposition | Defines the `Content-Disposition` header field for the attachment.          |
-| fileName    | Defines the `filename` parameter of the "Content-Disposition" header field. |
-| description | Defines the `Content-Description` header field for the attachment.          |
+`description`
+: Defines the `Content-Description` header field for the attachment.
 
 For example:
 
@@ -132,12 +156,15 @@ See the {ref}`mail scope <config-mail>` section to learn more the mail server co
 
 ### AWS SES configuration
 
-As of version `23.06.0-edge` Nextflow supports [AWS SES](https://aws.amazon.com/ses/) native API as an alternative
+:::{versionadded} 23.06.0-edge
+:::
+
+Nextflow supports [AWS SES](https://aws.amazon.com/ses/) native API as an alternative
 provider to send emails in place of SMTP server.
 
 To enable this feature add the following environment variable in the launching environment:
 
-```
+```bash
 export NXF_ENABLE_AWS_SES=true
 ```
 

docs/process.md

@@ -662,14 +662,6 @@ ciao
 hello
 ```
 
-(process-input-set)=
-
-### Input type `set`
-
-:::{deprecated} 19.08.1-edge
-Use `tuple` instead.
-:::
-
 (process-input-tuple)=
 
 ### Input type `tuple`
@@ -679,20 +671,20 @@ The `tuple` qualifier allows you to group multiple values into a single input de
 ```groovy
 process tupleExample {
     input:
-    tuple val(x), path('latin.txt')
+    tuple val(x), path('input.txt')
 
     """
     echo "Processing $x"
-    cat - latin.txt > copy
+    cat input.txt > copy
     """
 }
 
 workflow {
-  Channel.of( [1, 'alpha'], [2, 'beta'], [3, 'delta'] ) | tupleExample
+  Channel.of( [1, 'alpha.txt'], [2, 'beta.txt'], [3, 'delta.txt'] ) | tupleExample
 }
 ```
 
-In the above example, the `tuple` input consists of the value `x` and the file `latin.txt`.
+In the above example, the `tuple` input consists of the value `x` and the file `input.txt`.
 
 A `tuple` definition may contain any of the following qualifiers, as previously described: `val`, `env`, `path` and `stdin`. Files specified with the `path` qualifier are treated exactly the same as standalone `path` inputs.
 
@@ -958,19 +950,19 @@ Available options:
   even if only one file is produced.
 
 `followLinks`
-: When `true` target files are return in place of any matching symlink (default: `true`)
+: When `true`, target files are returned in place of any matching symlink (default: `true`)
 
 `glob`
-: When `true` the specified name is interpreted as a glob pattern (default: `true`)
+: When `true`, the specified name is interpreted as a glob pattern (default: `true`)
 
 `hidden`
-: When `true` hidden files are included in the matching output files (default: `false`)
+: When `true`, hidden files are included in the matching output files (default: `false`)
 
 `includeInputs`
-: When `true` any input files matching an output file glob pattern are included.
+: When `true` and the output path is a glob pattern, any input files matching the pattern are also included in the output (default: `false`)
 
 `maxDepth`
-: Maximum number of directory levels to visit (default: no limit)
+: Maximum number of directory levels to visit (default: no limit).
 
 `type`
 : Type of paths returned, either `file`, `dir` or `any` (default: `any`, or `file` if the specified file name pattern contains a double star (`**`))
@@ -1094,14 +1086,6 @@ workflow {
 }
 ```
 
-(process-set)=
-
-### Output type `set`
-
-:::{deprecated} 19.08.1-edge
-Use `tuple` instead.
-:::
-
 (process-out-tuple)=
 
 ### Output type `tuple`

docs/sharing.md

@@ -489,6 +489,10 @@ For example, shebang definitions `#!/usr/bin/python` and `#!/usr/local/bin/pytho
 
 Nextflow will automatically add the `bin/` directory to the `PATH` environment variable, and the scripts will automatically be accessible in your pipeline without the need to specify an absolute path to invoke them.
 
+### Utility code
+
+Any Groovy scripts or JAR files in the `lib` directory will be automatically loaded and made available to your pipeline scripts. The `lib` directory is a useful way to provide utility code or external libraries without cluttering the pipeline scripts.
+
 ### System environment
 
 Any environment variable that may be required by the tools in your pipeline can be defined in the `nextflow.config` file by using the `env` scope and including it in the root directory of your project. For example:

docs/tracing.md

@@ -1,6 +1,6 @@
 (tracing-page)=
 
-# Tracing & visualisation
+# Reports
 
 (execution-log)=