Skip to content

Comments

Add pipeline run attempt configuration and notification support#712

Merged
yruslan merged 3 commits intomainfrom
copilot/add-pipeline-run-attempt-support
Feb 23, 2026
Merged

Add pipeline run attempt configuration and notification support#712
yruslan merged 3 commits intomainfrom
copilot/add-pipeline-run-attempt-support

Conversation

Copy link
Contributor

Copilot AI commented Feb 23, 2026

Adds configuration and notification support for tracking pipeline run attempts to enable auto-retry automation.

Configuration

Added two new runtime configuration properties:

pramen {
  runtime.attempt = 1          # Current attempt number
  runtime.max.attempts = 5     # Maximum attempts allowed
}

Both default to 1 for backward compatibility.

Data Flow

  • RuntimeConfigRuntimeInfoPipelineNotificationBuilder
  • Attempt information propagates through the execution pipeline to notifications

Notification Changes

Email notifications now include attempt information when maxAttempts > 1:

Execution for the run date 2026-02-23 (attempt 2 of 5).

When maxAttempts = 1 (default), attempt information is omitted to avoid noise.

Implementation

  • Extended RuntimeConfig and RuntimeInfo case classes with attempt and maxAttempts fields
  • Modified PipelineNotificationBuilderHtml.renderExecutionInfo() to conditionally render attempt details
  • Updated test factories and added coverage for new functionality

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • fastdl.mongodb.org
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java -Dbasedir=/home/REDACTED/work/pramen/pramen/pramen/core -Xmx1024m org.scalatest.tools.Runner -R /home/REDACTED/work/pramen/pramen/pramen/core/target/classes /home/REDACTED/work/pramen/pramen/pramen/core/target/test-classes -q (?<!LongSuite) -o -fWDF /home/REDACTED/work/pramen/pramen/pramen/core/target/surefire-reports/TestSuite.txt -u /home/REDACTED/work/pramen/pramen/pramen/core/target/surefire-reports/. (dns block)
  • packages.confluent.io
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -classpath /usr/share/apache-maven-3.9.12/boot/plexus-classworlds-2.9.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.12/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.12 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.12/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/pramen/pramen/pramen org.codehaus.plexus.classworlds.launcher.Launcher test -pl core -Dtest=RuntimeConfigSuite -DfailIfNoTests=false (dns block)
    • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -classpath /usr/share/apache-maven-3.9.12/boot/plexus-classworlds-2.9.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.12/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.12 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.12/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/pramen/pramen/pramen org.codehaus.plexus.classworlds.launcher.Launcher clean install -DskipTests (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Add support for specifying the attempt of a pipeline run in configuration and in notifications</issue_title>
<issue_description>## Background
For auto-retry automation purposes.

Feature

Add support for specifying the attempt of a pipeline run in configuration and in notifications.

Example [Optional]

Email notification should contain the attempt number if the maximum number of attempts is bigger than 1.

Hi,

This is a notification from Pramen for abcd/TEST_PIPELINE on uat. The job has succeeded with warnings. Execution for the run date 2026-02-23. Application ID: spark-application-1234567890123.

Proposed Solution [Optional]

pramen {
  runtime.attempt = 1
  runtime.max.attempts = 5
}

</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits February 23, 2026 08:23
…d RuntimeInfo

Co-authored-by: yruslan <4082463+yruslan@users.noreply.github.com>
Co-authored-by: yruslan <4082463+yruslan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for specifying pipeline run attempt in configuration Add pipeline run attempt configuration and notification support Feb 23, 2026
Copilot AI requested a review from yruslan February 23, 2026 08:35
@github-actions
Copy link

Unit Test Coverage

Overall Project 84.37% 🍏
Files changed 100% 🍏

Module Coverage
pramen:core Jacoco Report 86.32% 🍏
Files
Module File Coverage
pramen:core Jacoco Report RuntimeConfig.scala 92% 🍏
PipelineNotificationBuilderHtml.scala 90.91% 🍏
PipelineStateImpl.scala 90.39% 🍏

@yruslan yruslan marked this pull request as ready for review February 23, 2026 09:13
@yruslan yruslan merged commit 9dd48f9 into main Feb 23, 2026
6 checks passed
@yruslan yruslan deleted the copilot/add-pipeline-run-attempt-support branch February 23, 2026 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for specifying the attempt of a pipeline run in configuration and in notifications

2 participants