Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new way to generate api docs #2525

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 37 additions & 27 deletions .github/workflows/tools-api-docs-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@ name: nf-core/tools release API docs
on:
release:
types: [published]
push:
branches: dev
workflow_dispatch:
inputs:
release_version:
description: "Release version"
required: true
default: "dev"

# Cancel if a newer run is started
concurrency:
Expand All @@ -10,39 +18,41 @@ concurrency:

jobs:
api-docs:
name: Build & push Sphinx API docs
# checkout the nf-core/website repo, generate the docs with lazydocs and copy to `src/content/tools/$release_version` and open a PR with the changes on the website repo
runs-on: ubuntu-latest
strategy:
matrix:
dir:
- latest
- ${{ github.event.release.tag_name }}
steps:
- name: Check out source-code repository
- uses: actions/checkout@v3
with:
token: ${{ secrets.nf_core_bot_auth_token }}

- name: Checkout nf-core/website
uses: actions/checkout@v3
with:
repository: nf-core/website
token: ${{ secrets.nf_core_bot_auth_token }}
path: website

- name: Set up Python 3.11
- name: setup python
uses: actions/setup-python@v4
with:
python-version: 3.11
python-version: 3.12
cache: "pip"

- name: Install python dependencies
run: |
pip install --upgrade pip
pip install -r ./docs/api/requirements.txt
pip install .
- name: Install lazydocs
run: pip install lazydocs

- name: Build HTML docs
run: make --directory ./docs/api html
- name: Generate API docs
run: |
lazydocs nf-core/tools -o website/src/content/tools/${{github.ref_name}} --force \
--src-base-url="https://github.com/nf-core/tools/blob/${{github.ref_name}}/" \
--overview-file="index.md"

- name: Sync release docs
if: github.repository == 'nf-core/tools'
uses: SamKirkland/FTP-Deploy-Action@4.0.0
with:
server: ${{ secrets.ftp_server }}
username: ${{ secrets.ftp_username}}
password: ${{ secrets.ftp_password }}
local-dir: "./docs/api/_build/html/"
server-dir: ${{ secrets.ftp_server_old_site_dir }}/${{ matrix.dir }}/
protocol: ${{ secrets.ftp_protocol }}
port: ${{ secrets.ftp_port }}
- name: Commit and push changes
run: |
cd website
git config user.email "core@nf-co.re"
git config user.name "nf-core-bot"
git checkout -b update-tools-api-docs
git add src/content/tools/${{github.ref_name}}
git remote add nf-core-website https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/nf-core/website.git
git push nf-core-website update-tools-api-docs
4 changes: 2 additions & 2 deletions docs/api/_src/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,15 @@
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
language = "en"

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

# The name of the Pygments (syntax highlighting) style to use.
# pygments_style = None
pygments_style = "sphinx"


# -- Options for HTML output -------------------------------------------------
Expand Down
28 changes: 23 additions & 5 deletions nf_core/download.py
Original file line number Diff line number Diff line change
Expand Up @@ -754,33 +754,49 @@ def find_container_images(self, workflow_directory):
self.containers = self.prioritize_direct_download(previous_findings + config_findings + module_findings)

def rectify_raw_container_matches(self, raw_findings):
"""Helper function to rectify the raw extracted container matches into fully qualified container names.
"""
Helper function to rectify the raw extracted container matches into fully qualified container names.
If multiple containers are found, any prefixed with http for direct download is prioritized

Example syntax:

Early DSL2:

.. code-block:: groovy

if (workflow.containerEngine == 'singularity' && !params.singularity_pull_docker_container) {
container "https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0"
container 'https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0'
} else {
container "quay.io/biocontainers/fastqc:0.11.9--0"
container 'quay.io/biocontainers/fastqc:0.11.9--0'
}

Later DSL2:

.. code-block:: groovy

container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0' :
'biocontainers/fastqc:0.11.9--0' }"


Later DSL2, variable is being used:

.. code-block:: groovy

container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
"https://depot.galaxyproject.org/singularity/${container_id}" :
"quay.io/biocontainers/${container_id}" }"
'https://depot.galaxyproject.org/singularity/${container_id}' :
'quay.io/biocontainers/${container_id}' }"

container_id = 'mulled-v2-1fa26d1ce03c295fe2fdcf85831a92fbcbd7e8c2:afaaa4c6f5b308b4b6aa2dd8e99e1466b2a6b0cd-0'

DSL1 / Special case DSL2:

.. code-block:: groovy

container "nfcore/cellranger:6.0.2"

"""

cleaned_matches = []

# Thanks Stack Overflow for the regex: https://stackoverflow.com/a/3809435/713980
Expand Down Expand Up @@ -830,6 +846,7 @@ def rectify_raw_container_matches(self, raw_findings):
or a plain URL like in the old DSL2 convention

"""

direct_match = re.match(either_url_or_docker, container_value.strip())
if direct_match:
cleaned_matches.append(direct_match.group(0))
Expand All @@ -852,6 +869,7 @@ def rectify_raw_container_matches(self, raw_findings):
example result:
['https://depot.galaxyproject.org/singularity/scanpy:1.7.2--pyhdfd78af_0', 'biocontainers/scanpy:1.7.2--pyhdfd78af_0']
"""

container_value_defs = [
capture for _, capture in container_value_defs[:] if not capture in ["singularity", "apptainer"]
]
Expand Down
1 change: 1 addition & 0 deletions nf_core/lint/multiqc_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ def multiqc_config(self):

export_plots: true
"""

passed = []
failed = []

Expand Down
1 change: 1 addition & 0 deletions nf_core/lint/nextflow_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ def nextflow_config(self):
* A ``test`` configuration profile should exist.

"""

passed = []
warned = []
failed = []
Expand Down
Loading