New user

User-Specific Pilot Configuration

Different PanDA users may have distinct requirements and preferences for how the pilot should operate. To support this flexibility, the pilot implements a plugin-based architecture that separates user-specific functionality from the core pilot framework. This design allows users to customize pilot behavior without modifying the main codebase.

To create a new user configuration:

1. Create a new directory under pilot/users/ corresponding to the new user.
2. The name of this directory will serve as the user label.
3. When launching the pilot, specify this label using the --pilot-user NAME option.

This mechanism ensures that user-specific logic is cleanly encapsulated and easily maintained, while the core pilot framework remains consistent and stable.

Actual Configuration

The pilot utilizes a standard configuration file located at pilot/util/default.cfg. This file defines a wide range of parameters (for example, Pilot.pandaserver) whose values typically remain static across runs.

In addition to the configuration file, the pilot can be launched with various command-line options that determine its behavior at startup. These options are fixed for the duration of the pilot’s execution and cannot be modified while it is running. For a complete list and description of available options, refer to the Pilot Options documentation

Implementation

The pilot ships with a generic user plugin that serves as a reference implementation. It provides default hooks the pilot expects and demonstrates how to customize behavior safely without touching the core framework. A user should copy this plugin to a new directory (e.g., pilot/users/atlas/) and adapt only what’s necessary.

Directory layout (generic plugin)

generic/
  __init__.py
  common.py
  container.py
  copytool_definitions.py
  cpu.py
  diagnose.py
  jobdata.py
  jobmetrics.py
  loopingjob_definitions.py
  memory.py
  monitoring.py
  proxy.py
  setup.py
  utilities.py

Module overview

• common.py - Payload and workflow helpers. Central helpers that the pilot calls to prepare and run the user payload. Customize when you need to change how the payload command is assembled, what gets validated, or which auxiliary utilities run. Typical responsibilities:
  • Build the payload command (including proxy export, transform setup, utility commands).
  • Validate job inputs/outputs; perform sanity checks and cleanup of redundant files.
  • Inject utility commands to run before/with/after the payload (e.g., sidecar monitors).
  • Small policy/gating decisions (e.g., time floors, debug toggles).
• container.py - Containerization logic. Customize if you need site-specific images, mount layouts, GPU/CPU flags, or environment preparation inside the container. Everything related to running payloads in a container:
  • Decide whether to use a container, and if so, which image/platform to select.
  • Build the full container wrapper command (mounts, env, user proxy handling, workdir, entrypoint).
  • Provide stage-in/stage-out container helpers and integration with environment setup tools (e.g., asetup/ALRB).
• copytool_definitions.py - Copy tool support. Extend if you add or tune copy tools or need alternate finalization rules/locations. Housekeeping for data movement:
• Helper functions used by copy tools (e.g., moving files to their final destination, path resolution).
• cpu.py - CPU/core accounting. Adapt if your site has special core counting rules or needs to record additional CPU metrics. Lightweight helpers for CPU usage:
  • Determine core counts actually used by the payload and reconcile with requested counts.
• diagnose.py - Payload diagnostics. Extend if you want richer pattern detection, site-specific error signatures, or tailored triage messages. Post-run analysis utilities:
  • Parse and interpret payload logs, extract relevant pilot log snippets, and return focused diagnostics.
• jobdata.py - Job parameter processing. Customize if you need to rewrite parameters, enforce naming schemes, or inject site defaults. Input parameter filters and post-processing:
  • Pre-filter job parameters before execution.
  • Post-filter after job preparation to align with site conventions or pilot expectations.
  • Provide clear failure semantics when job data is missing.
• jobmetrics.py — Metrics emission. Extend to add site-specific metrics or to integrate with local accounting. Compose and return job metrics (e.g., CPU, I/O, memory) for monitoring/accounting systems.
• loopingjob_definitions.py — Looping job policy. Adjust thresholds or file pruning rules to match your operational policy. Controls detection and handling of looping (runaway) jobs:
  • Decide if detection is allowed and what cleanup/removal of unwanted files to perform.
• memory.py — Memory-use checks. Tune for local memory enforcement rules or site alerting. Guardrails for memory usage:
  • Decide if memory verification is enabled and how usage is measured/reported.
• monitoring.py — Lightweight monitors. Add site telemetry or export small heartbeat files, but keep this lightweight. Hooks for fast/periodic monitoring tasks to run alongside the payload.
• proxy.py — User proxy management. Customize for local trust stores, robot proxies, or VO-specific role logic. Everything around X.509/VOMS proxies:
  • Verify proxy validity and attributes (e.g., VO role), retrieve/refresh proxies when needed, and expose a proxy dictionary.
• setup.py — Environment & transform setup. Modify if your site uses different transform distribution, mirror priorities, or alternative setup stacks. Pre-payload setup helpers:
  • Download/resolve analysis transforms, select valid base URLs, and integrate with environment tools (e.g., ALRB).
  • Determine setup timing and priorities from queue/site configuration.
• utilities.py — Small utilities. Defines utility tools that can be run before, during of after the payload. Miscellaneous helpers:
  • Pre-cleanup of workspace, CPU architecture probing, and other small cross-cutting helpers.
• __init__.py — Package marker. Declares the plugin package; no user customization normally required.

For a more advanced implementation, see especially the ATLAS plugin.

Using the generic plugin as a starting point

1. Copy the generic plugin:

• cp -r pilot/users/generic pilot/users/<your_user_label>

2. Rename and register: The directory name becomes the user label (e.g., atlas).
3. Launch the pilot with your label:

• python pilot.py --pilot-user <your_user_label> [other options]

4. Customize minimally:

• Start with common.py (payload command assembly) and container.py (if containerized).
• Adjust proxy.py only if your VO/role handling differs.
• Add metrics/diagnostics in jobmetrics.py (not strictly needed) and diagnose.py as needed.
• Keep changes localized; avoid editing the core pilot.

Guidance and best practices

1. Keep policy local: Put site/VO tweaks in the plugin, not the core.
2. Prefer composition: Add small helpers over large rewrites; keep diffs to the generic version reviewable.
3. Test incrementally: Validate common.py changes first (payload command), then container and data movement.
4. Log clearly: Diagnostics in diagnose.py should point operators to actionable causes.

Pull requests

When a pilot version has been tested and is ready for deployment, please make a pull request to the next branch and not directly to master. The next branch is merged with other versions in development before release. Also, cron jobs are automatically creating pilot tarballs from the master branch so only well-tested next versions will be merged with the master branch to avoid unintentional releases.

Python versions

Please only use late Python versions. As of October 2025, the minimum Python version that is supported is 3.9, but is likely to be moved to a later version sooner rather than later. After a pull request, we automatically run pylint, flake8 and unit tests. The code must have a minimum pylint score of 8 to pass. We currently run the tests under Python versions 3.9, 3.11 and 3.12.