Skip to content

[Feature]: HTCondorCEBackend — grid CE over the shared _htcondor core #23

Description

@aldbr

User Story

As the WMS consumer,
I want an HTCondorCEBackend submitting to an HTCondor-CE with IDTOKENS and spooled file
transfer,
So that the second major grid-CE path exists on the new contract, including the destructive-fetch
semantics that motivated IC-ADR-002.

Feature Description

  • HTCondorCEBackend: JobBackend + OutputRetriever + Cancellable.
    Deliberately no LoadReporter (DIRAC's getCEStatus is an explicit "not supported") and no
    live diagnostics — capability absence is the point, and the integration suite must show the
    narrowing works.
  • Reuses the _htcondor core (submit description, ClassAd parsing, state
    map); differs in transport/auth: remote schedd (-pool/-remote) or CE submission, grid
    universe routing, -spool.
  • Token + proxy together per IC-ADR-003 (its Motivation 7 / all-of kinds): SCITOKENS/IDTOKENS
    channel auth (token file + _CONDOR_* env injection, audience <ce>:9619) plus the X.509
    proxy materialised into the submit description (use_x509userproxy = true) — consumed
    site-side (VOMS attributes → APEL accounting), including the
    _CONDOR_DELEGATE_JOB_GSI_CREDENTIALS=false workaround (HTCONDOR-2904). The pure-IDTOKENS
    basic stack config exercises the token path; a proxy-bearing config is the ADR-002 auth axis.
  • Destructive retrieval: condor_transfer_data then declared destructive=True
    (HTCondor ≥ 25.8: completed spooled job leaves the queue); evaluate leave_in_queue as the
    re-fetchability override and record the decision. Coarse bounded materialisation (submit-time
    transfer_output_files allow-list, transfer timeout, post-hoc size/count check).
  • Phase 2 wiring: contract suite against the htcondor stack's CE path (markers
    remote and htcondor); the fetch-twice destructive_fetch test is the flagship regression
    (versions axis [lts, latest] makes 25.8-class changes red on the Renovate PR).

Definition of Done

  • Contract suite green against the htcondor stack CE path with IDTOKENS
  • Fetch-twice test asserts the declared one-shot semantics; leave_in_queue decision recorded
  • isinstance(be, LoadReporter) is False and the suite's capability-skip path exercises it
  • Kill path green; per-id outcomes on partial batches

Alternatives Considered

  • Expressing it as BatchBackend(HTCondorCETransport, HTCondorScheduler) — open ADR question;
    blocked on a transport-level "destructive get" notion. Start monolithic, revisit (share only
    _htcondor functions, never a subtype).
  • GSI/VOMS auth — out of scope; IDTOKENS-first per ADR-002 §4.

Related Issues

Blocked by: # (_htcondor), #. Validated by: #5 / htcondor stack.

Additional Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions