New user

User-Specific Pilot Configuration

Different PanDA users may have distinct requirements and preferences for how the pilot should operate. To support this flexibility, the pilot implements a plugin-based architecture that separates user-specific functionality from the core pilot framework. This design allows users to customize pilot behavior without modifying the main codebase.

To create a new user configuration:

1. Create a new directory under pilot/users/ corresponding to the new user.
2. The name of this directory will serve as the user label.
3. When launching the pilot, specify this label using the --pilot-user NAME option.

This mechanism ensures that user-specific logic is cleanly encapsulated and easily maintained, while the core pilot framework remains consistent and stable.

Actual Configuration

The pilot utilizes a standard configuration file located at pilot/util/default.cfg. This file defines a wide range of parameters (for example, Pilot.pandaserver) whose values typically remain static across runs.

In addition to the configuration file, the pilot can be launched with various command-line options that determine its behavior at startup. These options are fixed for the duration of the pilot’s execution and cannot be modified while it is running. For a complete list and description of available options, refer to the Pilot Options documentation

Implementation

The pilot ships with a generic user plugin that serves as a reference implementation. It provides default hooks the pilot expects and demonstrates how to customize behavior safely without touching the core framework. A user should copy this plugin to a new directory (e.g., pilot/users/atlas/) and adapt only what’s necessary.

Directory layout (generic plugin)

generic/
  __init__.py
  common.py

  container.py
  copytool_definitions.py
  cpu.py
  diagnose.py
  jobdata.py
  jobmetrics.py
  loopingjob_definitions.py
  memory.py
  monitoring.py
  proxy.py
  setup.py
  utilities.py

Module overview

• **common.py** - Payload and workflow helpers. Central helpers that the pilot calls to prepare and run the user payload. Customize when you need to change how the payload command is assembled, what gets validated, or which auxiliary utilities run. Typical responsibilities:
  • Build the payload command (including proxy export, transform setup, utility commands).
  • Validate job inputs/outputs; perform sanity checks and cleanup of redundant files.
  • Inject utility commands to run before/with/after the payload (e.g., sidecar monitors).
  • Small policy/gating decisions (e.g., time floors, debug toggles).
• **container.py** - Containerization logic. Customize if you need site-specific images, mount layouts, GPU/CPU flags, or environment preparation inside the container. Everything related to running payloads in a container:
  • Decide whether to use a container, and if so, which image/platform to select.
  • Build the full container wrapper command (mounts, env, user proxy handling, workdir, entrypoint).
  • Provide stage-in/stage-out container helpers and integration with environment setup tools (e.g., asetup/ALRB).
• **copytool_definitions.py** - Copy tool support. Extend if you add or tune copy tools or need alternate finalization rules/locations. Housekeeping for data movement:
• Helper functions used by copy tools (e.g., moving files to their final destination, path resolution).
• **cpu.py** - CPU/core accounting. Adapt if your site has special core counting rules or needs to record additional CPU metrics. Lightweight helpers for CPU usage:
  • Determine core counts actually used by the payload and reconcile with requested counts.
• **diagnose.py** - Payload diagnostics. Extend if you want richer pattern detection, site-specific error signatures, or tailored triage messages. Post-run analysis utilities:
  • Parse and interpret payload logs, extract relevant pilot log snippets, and return focused diagnostics.