Skip to content

Commit

Permalink
Fix: Adding hash to pipelines processing step function execution name…
Browse files Browse the repository at this point in the history
…s to … (#641)

* Adding hash to pipelines processing step function execution names to prevent collisions

* Adding hash to pipelines processing step function execution names to prevent collisions

* Apply linting fixes

---------

Co-authored-by: Simon Kok <sbkok@users.noreply.github.com>
  • Loading branch information
avolip and sbkok committed Jul 24, 2023
1 parent 7a70a3b commit 7d25765
Showing 1 changed file with 7 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import json
import tempfile
from typing import Any, TypedDict
import hashlib
import yaml
from yaml.error import YAMLError

Expand Down Expand Up @@ -173,9 +174,12 @@ def start_executions(
# AWS Step Functions supports max 80 characters.
# Since the run_id equals 49 characters plus the dash, we have 30
# characters available. To ensure we don't run over, lets use a
# truncated version instead:
truncated_pipeline_name = full_pipeline_name[:30]
sfn_execution_name = f"{truncated_pipeline_name}-{run_id}"
# truncated version concatenated with an hash generated from
# the pipeline name
truncated_pipeline_name = full_pipeline_name[:24]
name_bytes_to_hash = bytes(full_pipeline_name + run_id, 'utf-8')
execution_unique_hash = hashlib.md5(name_bytes_to_hash).hexdigest()[:5]
sfn_execution_name = f"{truncated_pipeline_name}-{execution_unique_hash}-{run_id}"
sfn_client.start_execution(
stateMachineArn=PIPELINE_MANAGEMENT_STATEMACHINE,
name=sfn_execution_name,
Expand Down

0 comments on commit 7d25765

Please sign in to comment.