Skip to content

v0.1.0-rc1

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 18 Oct 15:27
· 7 commits to main since this release
b48c50e

Note

The first release candidate of v0.1.0. It requires pyTooling v10.0.0-rc3 or later and, for the extra sphinx, pyTooling.Sphinx v0.1.0-rc3 or later - both pre-releases on PyPI.

  • ✨ The package pyTooling.GitLab with its metadata - a sub-namespace of pyTooling, like pyTooling.GitHub.
  • ✨ GitLabError, the base exception; the requirements and the extras diagram and sphinx.
  • ✨ pyTooling.GitLab.PipelineFile: a pipeline configuration file with its inputs, stages, variables, jobs, their needs, rules, parallel instances and triggers, and the keys workflow, include and default; extends, !reference tags and defaults are merged into the jobs as GitLab does; a file converts into the pyTooling.CI pipeline it defines, reading its local includes and child pipelines (#7, #10, #12, #14, #16, #18, #19, #20, #21).
  • ✨ The pipeline run model: jobs, matrix instances, pipelines, downstream pipelines and pipeline groups as GitLab's REST API reports them, a client of that API for one project, and the dependencies a pipeline file's stages and needs imply (#8, #11, #13).
  • ✨ Pipeline traces: a pipeline read from GitLab's REST API as a software execution trace, for OTLP/JSON and Gantt charts (#15), and the program pytooling-gitlab writing them (#17).
  • ✨ The Sphinx domain glci: the directive glci:pipeline documents a pipeline file, glci:input and glci:variable its parameters; parameter tables, the interface, the dependencies and YAML excerpts summarize it, glci:pipeline-graph draws it (#22, #23, #24, #25, #26).
  • 📖 The documentation, structured as pyTooling.GitHub's: data models, Sphinx domain glci, visualization, program, reports.
  • ⚙️ The pipeline on pyTooling/Actions' CompletePipeline.

First release of pyTooling.GitLab, to work with GitLab CI/CD pipelines: a data model of pipeline configuration files (.gitlab-ci.yml, included files, child pipelines' files) and the pyTooling.CI pipeline they define; pipeline runs read from GitLab's REST API and their conversion into traces (OpenTelemetry's OTLP/JSON, Gantt charts), also by the program pytooling-gitlab; and the Sphinx domain glci, which documents pipeline files.

✨ New Features

  • pyTooling.GitLab: the package, holding its metadata only - __version__ = "0.1.0rc1", author, copyright, license, keywords, project, documentation and issue tracker URLs (#1).

  • pyTooling.GitLab.GitLabError (derived from pyTooling.CI.CIError): the base exception of the package (#5).

  • pyTooling.GitLab.PipelineFile (#7): a data model of a GitLab CI/CD configuration file (.gitlab-ci.yml, an included file, a child pipeline's file), read with ruamel.yaml - the counterpart of pyTooling.GitHub's WorkflowFile. This release reads a file's stages, variables, jobs, needs, rules and the keys workflow, include and default, and builds the pipeline the file defines.

    • PipelineFile.FromFile(path) reads a file; an optional header document holding spec (separated by ---) is accepted.
    • PipelineFile.DeclaredStages (the stages key as written) and PipelineFile.Stages (declared or the default build, test, deploy, always between .pre and .post).
    • Variable: a variable of the top-level variables or of a job - Value, Description, Options, Expand.
    • Job (#10): a top-level key is a job unless it is a global keyword (RESERVED_KEYS); a key starting with a dot is a hidden job. PipelineFile.Jobs and PipelineFile.HiddenJobs keep them apart, with JobCount, ContainsJob(), IterateJobs(), IterateHiddenJobs().
    • A job's Stage (test if it names none, checked against PipelineFile.Stages), Script / BeforeScript / AfterScript, When (a pyTooling.CI.RuleOutcome, #30), AllowFailure and AllowedExitCodes, Image, Services, Environment, Tags and its Variables (#10).
    • Need (#12): an entry of a job's needs, with Kind (enumeration NeedKind): a job of the same pipeline, a job of another pipeline of the parent-child hierarchy, a job of another project's pipeline, or another project's pipeline whose status the job mirrors. Job.Needs is None without a needs key and empty for needs: []; Job.NeededJobs and Need.NeededJob resolve needs of the same pipeline to the file's jobs.
    • FromFile checks that a job needs only jobs of the file (an optional need may name a missing one) and that the jobs don't need each other in a cycle (#12).
    • Rule (#14): a rule of rules, kept as written, not evaluated - Condition (if), Changes and CompareTo, Exists, and When, AllowFailure and Variables applying if it matches; Job.Rules.
    • Workflow (#14): the workflow key - the pipeline's Name and its Rules; PipelineFile.Workflow.
    • Include (#16): an entry of include - Kind (enumeration IncludeKind: local, project, remote, template, component), Location as written, Files and Ref of another project, the Inputs it passes and its Rules; PipelineFile.Includes. The included files aren't read.
    • Default (#16): the default key - Image, Services, Tags, BeforeScript, AfterScript, also read from the deprecated global keywords unless default sets them; PipelineFile.Default.
    • A job's keys are merged as GitLab merges them, before the job is built (#18): !reference tags are replaced by what they refer to (a referred list is spliced into a list, nesting up to MAX_REFERENCE_DEPTH = 10); extends deep-merges the extended jobs in order, then the job's own keys, per GitLab's merge rules (up to MAX_EXTENDS_DEPTH = 11 levels, a cycle is an error; Job.Extends keeps the names); a job gets the keys of default it doesn't set, unless inherit.default says otherwise. Every key keeps the line it is written at.
    • Job.Unresolved (#18): in a file including other files, a job may extend or refer to jobs of the included files, which aren't read; the job is built from what the file has, and Unresolved names what it lacks. A file including no files must have everything it refers to.
    • Parallel (#19): a job's parallel - a Count, or a Matrix; Combinations holds one combination per instance in GitLab's order, InstanceNames the names GitLab gives the instances (test 1/3, deploy: [aws, app1]); Job.Parallel, and Need.Parallel for needs:parallel:matrix.
    • Trigger (#19): a job's trigger - a child pipeline's files as Includes, or another project's Project and Branch; Strategy (enumeration TriggerStrategy: depend, mirror), IsChildPipeline; Job.Trigger. IncludeKind.Artifact and Include.ArtifactJob include a job's artifact (a dynamic child pipeline).
    • Input (#20): an input of the header's spec:inputs - Type (enumeration InputType), Default, Required, Description, Options, Regex, conditional Rules; PipelineFile.Inputs and PipelineFile.Description. A job's inputs (GitLab 18.10) are Inputs too; Job.Inputs. A default is checked against the type and the options.
    • A stage or a needed job interpolating an input ($[[ inputs.stage ]]) is known when the file is included only, so it isn't checked against the stages or the jobs (#20).
    • PipelineFile.ToPipeline(resolver=None, depth=None) (#21): the pyTooling.CI pipeline a file defines, as pyTooling maps GitLab CI onto it - a job becomes a DefinedJob, a job with parallel a DefinedMatrix of DefinedMatrixJobs, a trigger job a DefinedWorkflow (DefinedMatrixWorkflow per instance); DefinedPipeline is named by workflow:name or the file's name. Stage order implies needs unless needs replaces them; matrix instances are named as GitLab names them (test: [3.14, linux], test 1/3); every element links back to its job by DefinitionMixin.Definition. Each element carries one pyTooling.CI.Rule per rule of its job - the rule's if, its when (else the job's, else on_success), and changes, compare_to, exists, allow_failure, variables as key-value-pairs; a job with when but no rules gets one rule without condition; workflow:rules are the pipeline's rules (#30).
    • IncludeResolver(root) (#21): reads local includes from a repository's directory, each file once; PipelineFile.FromFile(path, resolver) then resolves extends and !reference into the included files. A trigger job's child pipeline is expanded into its workflow when the resolver reads its files, limited by depth.
    • Every element knows its Parent, its PipelineFile, its Line and its Location (.gitlab-ci.yml:42) (#10); PipelineFileError(GitLabError) carries Path and Line.
  • The pipeline run model (#8, #11, #13): what GitLab's REST API reports about jobs, pipelines and downstream pipelines.

    • Status: GitLab's 13 job and pipeline statuses; an unknown value is a GitLabError whose note lists the known values. Status.ToOutcome() maps a status to pyTooling.CI's outcome.
    • Source: the 20 pipeline sources of CI_PIPELINE_SOURCE.
    • StatusMixin (status, failure reason, URL) and JobMixin (identifier, stage, allow_failure, queued duration).
    • Job(pyTooling.CI.Job, StatusMixin, JobMixin) with runner and requested runner tags, Job.Parse(json); MatrixJob for an instance of parallel:matrix or parallel: N.
    • ProjectClient(pyTooling.REST.RESTClient) (#11): a client of GitLab's REST API for one project, given by its numeric identifier or its path; apiURL reaches a self-managed instance. GetJSONArray(resourcePath, pageSize=100) reads a list GitLab answers as a JSON array, following the Link: rel="next" pages.
    • Pipeline(pyTooling.CI.Pipeline, StatusMixin, PipelineContentsMixin) (#11): identifier, project number (IID), project identifier, source, reference, commit, IsTag, queued duration; Pipeline.Parse(pipeline, jobs). Jobs are added in the order GitLab created them, matrix instances grouped below a pyTooling.CI.Matrix; a pipeline without workflow:name is named by its number (#47177).
    • PipelineGroup(pyTooling.CI.PipelineGroup) (#11): a commit's pipelines (branch, merge request, tag, schedule), ByGitReference(), PipelineGroup.Parse(pipelines, jobs, sha).
    • DownstreamPipeline(pyTooling.CI.Workflow, StatusMixin, JobMixin, PipelineContentsMixin) (#13): a trigger job (GitLab's bridge) and the pipeline it started, reporting the downstream pipeline's own Status, URL and times, DownstreamID, DownstreamProjectID, IsMultiProject, and the trigger job's TriggerStatus and TriggerURL. A child pipeline holds its own jobs, matrices and downstream pipelines; a multi-project pipeline names its project as Reference. MatrixDownstreamPipeline for a trigger job with parallel:matrix.
    • Pipeline.Parse(pipeline, jobs, bridges, downstream) reads downstream pipelines; a recording is read with Pipeline.Parse(**recording) (#13).
    • PipelineContentsMixin.ApplyNeeds(stages, needs=None) (#13): adds the dependencies a pipeline file's stages: and needs: imply, which GitLab's REST API doesn't report.
  • pyTooling.GitLab.Tracing (#15): a GitLab pipeline as a software execution trace, like pyTooling.GitHub's Tracing.

    • PipelineTrace(pyTooling.Tracing.CI.PipelineTrace): Parse(pipeline, jobs, bridges, downstream) and FromPipeline(pipeline); a matrix (MatrixSpan) and a downstream pipeline (DownstreamSpan) group their jobs, a job (JobSpan) follows its waiting time (QueuedSpan), which is GitLab's reported queued_duration right before the job's start. A job that hasn't started has no timespan; a skipped job lasts no time.
    • GitLab: attribute keys of what only GitLab reports (gitlab.status, gitlab.job.stage, gitlab.runner.tags, gitlab.downstream.id, gitlab.trigger.status, ...) beside OpenTelemetry's CI/CD attributes.
    • PipelineReader(ProjectClient): ReadPayloads(pipelineID, downstream=True) reads a pipeline, its jobs, its trigger jobs and its child pipelines; ReadPipeline() returns the trace. HTTP 401/403/404 add a note what to check.
    • runnerCategory(span): a Gantt chart's category for a job - its first runner tag, else the runner's description.
  • pyTooling.GitLab.CLI (#17): the program pytooling-gitlab, like pytooling-github - help, version and pipeline, which reads a pipeline into a trace and writes --trace-file (OTLP/JSON) and --gantt (PNG/SVG/PDF, colored per runner). --gitlab-project and --gitlab-pipeline-id default to $CI_PROJECT_ID and $CI_PIPELINE_ID, --gitlab-api-url to $CI_API_V4_URL (else gitlab.com); the token is --token, else $GITLAB_TOKEN. Console script in setup.py.

    pytooling-gitlab pipeline --gitlab-project=gitlab-org/gitlab-development-kit --gitlab-pipeline-id=2954836704 --trace-file=report/Pipeline.otlp.json --gantt=report/Pipeline.png
    
  • pyTooling.GitLab.Sphinx (#22): the Sphinx extension registering the domain glci for GitLab CI/CD pipeline files - the counterpart of pyTooling.GitHub's ghactions; installed with the extra sphinx.

    • Directive glci:pipeline: reads a pipeline file and makes it the document's current pipeline, with a target, an index entry and a :ref: label (PIPELINE/<name>). The file is named by :file:, or looked up in glci_pipeline_directory (<name>.yml, <name>.yaml, a component's <name>/template.yml); local includes and child pipelines are read from glci_repository_root.
    • Role :glci:pipeline:, also through :any:; configuration values glci_server, glci_project, glci_repository_root, glci_pipeline_directory, glci_ref, glci_label_prefix; warnings glci.pipeline and glci.duplicate; parallel and incremental builds.
    • Directives glci:input and glci:variable (#23): one parameter of the current pipeline - an input of spec:inputs, a default variable of the top-level variables - as a section whose fields show the file's facts (type, required, default value, possible values, pattern), then the hand-written fields; roles :glci:input: and :glci:variable:, labels PIPELINE/<Pipeline>/Input/<name> and .../Variable/<name>. Warning glci.drift: an entry of a parameter the file doesn't have, or a hand-written field repeating a fact of the file.
    • pyTooling.GitLab.Sphinx.Reference (#24): glci:parameter-table (a table per kind - inputs, variables - each name linked to its entry), glci:interface (the contract with an including pipeline: required inputs, variables, stages, jobs, hidden jobs) and glci:autoinputs (an entry for every input the document doesn't document); an input without an entry is a glci.drift warning.
    • glci:dependencies and glci:yaml (#25): what the current pipeline file uses as a nested list - includes (local files read with their own dependencies), child and other projects' pipelines, images and service images - linked to GitLab at glci_ref, merged with a hand-written list; an excerpt of the file, a section or a job, numbered as in the file, its caption linking to the lines on GitLab.
    • pyTooling.GitLab.Sphinx.Graph (#26): PipelineDotGraph draws the pipeline a file defines (ToPipeline() → ToGraph()) in DOT - jobs with their stage, child pipelines expanded into clusters as deep as depth allows, another project's pipeline a white leaf, jobs whose rules have conditions or with when: manual dashed, matrices as clusters of their instances named as GitLab names them; edges are the needs, or the stages, reduced. Directive glci:pipeline-graph (:depth:, :direction:, :reduce:, :link:, ...); in HTML a trigger job links to the page documenting its child pipeline.
  • Packaging (setup.py, pyproject.toml) as for pyTooling.GitHub, a namespace package next to pyTooling: Python 3.11 to 3.15, development status pre-alpha, requirements pyTooling[terminal] >= 10.0.0rc3, < 11.0 (#9) and ruamel.yaml ~= 0.19.1; extras diagram (matplotlib) and sphinx (pyTooling.Sphinx) (#1, #5). The wheel has no pyTooling/__init__.py, so the pyTooling namespace stays open for pyTooling and its sibling distributions.

📖 Documentation

  • doc/: structured as pyTooling.GitHub's (#1, #5): a landing page with the badges, the features (#31), contributors and license; News, Installation, Dependencies, Competing Solutions (python-gitlab and others), Glossary; a page per feature; the unit test, code coverage, documentation coverage and type checking report pages; License (Apache 2.0) and Doc-License (CC-BY 4.0); the Python class reference by autoapi.
  • README.md with the badges, the package's purpose, the features (#31), contributors and license; LICENSE.md (Apache 2.0).
  • News lists the base exception GitLabError (#27) and describes v0.1.0's scope: the data models, the Sphinx domain glci, the pipeline graph and trace, and the program (#31).
  • Data Models → Pipeline File (doc/Data/PipelineFile.rst, label DATA/PipelineFile): the tree, the header document, stages, variables, jobs, dependencies between jobs, rules and workflow, includes and defaults, extends and references, parallel jobs and triggers, inputs, building a pipeline, source lines (#7, #10, #12, #14, #16, #18, #19, #20, #21).
  • Data Models → Pipeline Run (label DATA/PipelineRun): the tree, matrices and their names, the order of jobs, several pipelines per commit (#11); downstream pipelines, stages and needs (#13).
  • Dependencies § Unit Testing lists the recorded pipelines used as test data; NOTICE.md, linked from the README's License section (#11).
  • Pipeline Graph (VIS/PipelineGraph), rendering an example pipeline with a matrix, a manual job, a child pipeline and another project's pipeline (#26).
  • pytooling-gitlab (CLI): commands, failures, the pipeline command, formats; Dependencies § Application Testing (Optional) (#17).
  • Pipeline Trace Diagram (VIS/PipelineTrace): reading a pipeline, timespans and attributes, what differs from a GitHub run, the Gantt chart, with a figure drawn from a recorded pipeline (#15).
  • glci Domain: Overview (configuration, labels, warnings, extending the domain) and Pipelines and Parameters; doc/conf.py enables the extension (#22); parameters, their labels and the drift warnings (#23); Summaries (parameter tables, the interface), glci:autoinputs (#24); dependencies and YAML excerpts (#25).
  • README.md and the landing page show the Codacy badges for code quality and coverage (#29).
  • The documentation installs pyTooling.Sphinx[reports] ~= 0.1.0rc3 from PyPI, the unit tests pyTooling.Sphinx ~= 0.1.0rc3; the Dependencies page names the version (#9).

🧪 Unit Tests

  • tests/unit/Run.py: testsuites Status and source and Jobs (#8); Project client, Pipelines and Recorded pipelines, with two pipelines of public MIT projects on gitlab.com in tests/data/Runs/, reduced to the fields GitLab reports about pipelines and jobs (#11); Downstream pipelines and Stage dependencies (#13).
  • tests/unit/PipelineFile.py with fixtures in tests/data/PipelineFiles/: the file, declared and default stages, the header, empty files, the variables of every form, constructors and their parameter checks (#7); testsuites Jobs, Parameter checks of jobs, Malformed jobs (#10); Needs, Parameter checks of needs, Malformed needs (#12); Rules and workflow, Parameter checks of rules and workflow, Malformed rules and workflow (#14); Includes and default, Parameter checks of includes and default, Malformed includes and default (#16); Extends, references and default, Malformed extends and references (#18); Parallel jobs and triggers, Parameter checks of parallel and trigger, Malformed parallel and trigger (#19); Inputs, Parameter checks of inputs, Malformed inputs (#20); Pipeline built from a file (with Rules and Rules: when and key-value-pairs, #30), Include resolver, Defined elements, Included files with fixture Repository/ (#21).
  • tests/unit/Package.py: testsuite Package metadata - the version parses as a semantic version, the URLs name the repository (#1, #3); testsuite Exceptions (#5).
  • tests/unit/CLI.py: the program in-process; tests/app/Program.py: the installed console script, including a public gitlab.com pipeline read without a token (#17).
  • tests/unit/Tracing.py: testsuites Attribute keys, Timespans (pipeline, job and its queue, matrix, child and downstream pipelines, runner category, the recorded pipeline) and Reader (with a faked urlopen) (#15).
  • tests/unit/Sphinx/Domain.py: testsuites Pipelines and Builds, each testcase building a small Sphinx project; skipped below Python 3.12 (Sphinx 9.1) (#22); tests/unit/Sphinx/Reference.py: Parameter tables, Interfaces, Automatic input entries, Undocumented inputs (#24); Dependency lists, YAML excerpts (#25); tests/unit/Sphinx/PipelineGraph.py: Nodes, Edges, Clusters, Parameters, Directive (#26).
  • tests/unit/requirements.txt: pytest-xdist ~= 3.8; pyTooling/Actions@r9 runs the unit tests with pytest-xdist workers, as [tool.coverage.run] sets patch = ["subprocess"], so Coverage.py measures the workers (#6, #32).

⚙️ GitHub Pipeline

  • .github/workflows/Pipeline.yml on pyTooling/Actions/.github/workflows/CompletePipeline.yml@r9, as in pyTooling.GitHub: unit tests on Python 3.11 to 3.15 on Linux, macOS, Windows and MSYS2 UCRT64, application tests (#17), packaging, documentation, static type check, Bandit, test reports, coverage to Codecov and Codacy, GitHub Pages, and on a tag the release and the upload to PyPI (#3).
  • dist/requirements.txt (wheel, twine), which the PyPI release job installs from the package artifact (#4).

Others

  • Repository files taken from pyTooling.GitHub: .editorconfig, .gitignore, .github/CODEOWNERS, .github/pull_request_template.md, run.ps1, the requirements files (#1).
  • f-strings without placeholder are plain strings: f"…" became "…" (2 strings in doc/conf.py); every string keeps its value (#28).

Related Issues and Pull-Requests

  • #1 - repository infrastructure.
  • #3 - pipeline on CompletePipeline.
  • #4 - dist/requirements.txt for the PyPI release job.
  • #5 - foundation: documentation skeleton, requirements, GitLabError.
  • #6 - pytest-xdist for the unit tests.
  • #7 - PipelineFile, part 1: the configuration file, its stages and variables.
  • #8 - Pipeline run model, part 1: status, pipeline source, jobs and matrix instances.
  • #9 - pyTooling.Sphinx and pyTooling from PyPI: latest pre-releases.
  • #10 - PipelineFile, part 2: jobs and hidden jobs.
  • #11 - Pipeline run model, part 2: project client, pipelines and pipeline groups.
  • #12 - PipelineFile, part 3: needs.
  • #13 - Pipeline run model, part 3: downstream pipelines and the dependencies of stages.
  • #15 - Pipeline traces: a GitLab pipeline as a software execution trace.
  • #17 - Program pytooling-gitlab: command pipeline.
  • #14 - PipelineFile, part 4: rules and the workflow key.
  • #16 - PipelineFile, part 5: includes and the default key.
  • #18 - PipelineFile, part 6: extends, !reference tags and the default applied.
  • #19 - PipelineFile, part 7: parallel jobs, matrices and triggers.
  • #20 - PipelineFile, part 8: inputs.
  • #21 - PipelineFile, part 9: the pipeline a file defines, and an include resolver.
  • #22 - Sphinx domain glci, part 1: the domain, pipelines and their configuration.
  • #23 - Sphinx domain glci, part 2: inputs and variables.
  • #24 - Sphinx domain glci, part 3: parameter tables, interface, automatic input entries.
  • #25 - Sphinx domain glci, part 4: dependencies and YAML excerpts.
  • #26 - Sphinx domain glci, part 5: the pipeline graph.
  • #27 - News lists GitLabError.
  • #28 - f-strings without placeholder are plain strings.
  • #30 - PipelineFile: the elements a file defines carry pyTooling.CI rules.
  • #29 - Codacy badges.
  • #31 - README, landing page and News: the features are implemented.
  • #32 - Coverage.py follows subprocesses: unit tests with pytest-xdist workers.

Published from Pipeline workflow triggered by github-actions[bot] on 2026-10-18 15:27:45 UTC.

This automatic release was created by pyTooling/Actions::PublishReleaseNotes.yml