Feat(x2a): collectArtifacts with HMAC authz#2403
Feat(x2a): collectArtifacts with HMAC authz#2403elai-shalev merged 3 commits intoredhat-developer:mainfrom
Conversation
Missing ChangesetsThe following package(s) are changed by this PR but do not have a changeset:
See CONTRIBUTING.md for more information about how to add changesets. Changed Packages
|
eloycoto
force-pushed
the
flpath-3341
branch
2 times, most recently
from
5e2699a to
048162a
Compare
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
|||||||||||||||||||||||||
PR Code Suggestions ✨Explore these optional code suggestions:
|
||||||||||||
|
Let's discuss that offline first |
sure, just saw the discussion on slack |
elai-shalev
left a comment
elai-shalev
left a comment
There was a problem hiding this comment.
overall looks good, some comments
| // Validate job age (3 hours = 10800 seconds) to prevent replay attacks | ||
| authHandler.validateJobAge(jobWithToken, 10800); |
There was a problem hiding this comment.
maybe lets make it configurable from the app config instead of hardcoding. the default value could still stay 10800
| // Use 256-bit token to match HMAC-SHA256 strength | ||
| const callbackToken = crypto.randomBytes(32).toString('hex'); |
There was a problem hiding this comment.
We can move this to common.ts as it is the same in modules.ts
not a must
| const bodyHash = this.computeBodyHash(rawBody); | ||
| this.logger.info( | ||
| `Signature validated for job ${jobId} (bodyHash: ${bodyHash})`, | ||
| ); | ||
| } |
There was a problem hiding this comment.
a full SHA-256 hash of the raw body is computed purely for an info log. With the 10MB body limit, this is a non-trivial cost on every request. The body length (rawBody.length) or a prefix of the already-validated signature would serve the same debugging purpose without the extra crypto work.
| const logs = await fetchJobLogs( | ||
| kubeService, | ||
| logger, | ||
| job.k8sJobName as string, |
There was a problem hiding this comment.
no need to cast 'as string' as the fetchJobLogs can handle null values gracefully
| @@ -16,7 +16,12 @@ | |||
|
|
|||
There was a problem hiding this comment.
Maybe lets add a test for replay attack prevention (job age validation)
the test suite covers signature validation but has no test for the 3-hour job-age window. A test with a startedAt set to 4 hours ago should return 401.
- Add a new auth system using HMAC fingerprints.
- Create SignatureValidator utility class for secure signature operations
- Update job tokens to 256-bit
- Take care of time-replay attacks(3 hours)
- Added test for all corner cases.
- Update openapi specs.
I tested with this script:
```
set -e
BASE_URL="${X2A_BASE_URL:-http://localhost:7007/api/x2a}"
AUTH_URL="${AUTH_URL:-http://localhost:7007/api/auth}"
PROJECT_ID="$1"
PHASE="${2:-init}"
DB_PATH="${X2A_DB_PATH:-./packages/backend/x2a.sqlite}"
if ! command -v jq &> /dev/null; then
echo "Error: jq is required. Install with: sudo dnf install jq"
exit 1
fi
if ! command -v sqlite3 &> /dev/null; then
echo "Error: sqlite3 is required. Install with: sudo dnf install sqlite"
exit 1
fi
echo ""
echo "Creating job..."
echo "==============="
echo "Authenticating with guest token..."
AUTH_RESPONSE=$(curl -s -X GET "$AUTH_URL/guest/refresh" \
-H "Content-Type: application/json" 2>&1)
TOKEN=$(echo "$AUTH_RESPONSE" | jq -r '.backstageIdentity.token // empty')
if [ -z "$TOKEN" ]; then
echo "✗ Failed to get authentication token"
echo "Response: $AUTH_RESPONSE"
echo ""
echo "Is the backend running at $AUTH_URL?"
exit 1
fi
echo "✓ Authenticated"
if [ -z "$PROJECT_ID" ]; then
echo "Creating new project..."
PROJECT_RESPONSE=$(curl -s -X POST "$BASE_URL/projects" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"name": "Test Project '"$(date +%s)"'",
"description": "Test project",
"abbreviation": "TP'"$(date +%s | tail -c 4)"'",
"sourceRepoUrl": "https://github.com/test/source",
"targetRepoUrl": "https://github.com/test/target",
"sourceRepoBranch": "main",
"targetRepoBranch": "main"
}')
PROJECT_ID=$(echo "$PROJECT_RESPONSE" | jq -r '.id // empty')
if [ -z "$PROJECT_ID" ]; then
echo "✗ Failed to create project"
echo "Response: $PROJECT_RESPONSE"
exit 1
fi
echo "✓ Project created: $PROJECT_ID"
fi
echo "Creating $PHASE job..."
if [ "$PHASE" != "init" ]; then
echo "Creating module for $PHASE phase..."
MODULE_RESPONSE=$(curl -s -X POST "$BASE_URL/projects/$PROJECT_ID/modules" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"name": "Test Module",
"sourcePath": "/cookbooks/test"
}')
MODULE_ID=$(echo "$MODULE_RESPONSE" | jq -r '.id // empty')
if [ -z "$MODULE_ID" ]; then
echo "✗ Failed to create module"
echo "Response: $MODULE_RESPONSE"
exit 1
fi
echo "✓ Module created: $MODULE_ID"
# Module-level job
JOB_RESPONSE=$(curl -s -X POST "$BASE_URL/projects/$PROJECT_ID/modules/$MODULE_ID/run" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"phase": "'"$PHASE"'",
"sourceRepoAuth": {"token": "test-token"},
"targetRepoAuth": {"token": "test-token"}
}')
else
# Project-level init job (no phase parameter needed)
JOB_RESPONSE=$(curl -s -X POST "$BASE_URL/projects/$PROJECT_ID/run" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"sourceRepoAuth": {"token": "test-token"},
"targetRepoAuth": {"token": "test-token"}
}')
fi
JOB_ID=$(echo "$JOB_RESPONSE" | jq -r '.jobId // empty')
if [ -z "$JOB_ID" ] || [ "$JOB_ID" = "null" ]; then
echo "✗ Failed to create job"
echo "Response: $JOB_RESPONSE"
exit 1
fi
echo "✓ Job created: $JOB_ID"
echo "Fetching callback token from database..."
if [ ! -f "$DB_PATH" ]; then
echo "✗ Database file not found at: $DB_PATH"
echo ""
echo "Make sure the backend is configured to use a file-based SQLite database."
echo "Check app-config.yaml: backend.database.connection should be a file path."
exit 1
fi
CALLBACK_TOKEN=$(sqlite3 "$DB_PATH" "SELECT callback_token FROM jobs WHERE id = '$JOB_ID';" 2>&1)
if [ -z "$CALLBACK_TOKEN" ]; then
echo "✗ Failed to retrieve callback token"
echo ""
echo "Query the database manually:"
echo " sqlite3 $DB_PATH \"SELECT id, phase, callback_token FROM jobs WHERE id = '$JOB_ID';\""
exit 1
fi
echo "✓ Token retrieved"
echo ""
echo "=================================="
echo "Job Created Successfully!"
echo "=================================="
echo ""
echo "Project ID: $PROJECT_ID"
echo "Job ID: $JOB_ID"
echo "Phase: $PHASE"
echo "Callback Token: $CALLBACK_TOKEN"
echo ""
echo "Test with UV (X2Ansible convertor CLI):"
echo "----------------------------------------"
echo "uv run app.py report \\"
echo " --url \"http://localhost:7007/api/x2a/projects/$PROJECT_ID/collectArtifacts?phase=$PHASE\" \\"
echo " --job-id \"$JOB_ID\" \\"
echo " --callback-token \"$CALLBACK_TOKEN\" \\"
echo " --artifacts \"migration_plan:https://storage.example/plan.md\""
echo ""
echo "" | sed "s/JOB_ID/$JOB_ID/g; s/CALLBACK_TOKEN/$CALLBACK_TOKEN/g; s/PROJECT_ID/$PROJECT_ID/g; s/PHASE/$PHASE/g"
```
Fix FLPATH-3341
Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
elai-shalev
left a comment
elai-shalev
left a comment
There was a problem hiding this comment.
Should the changes marked TODO here: https://github.com/redhat-developer/rhdh-plugins/blob/main/workspaces/x2a/plugins/x2a-backend/templates/x2a-job-script.sh#L14-L19
should be part of this PR? The job would fail without them?
|
* Feat(x2a): collectArtifacts with HMAC authz
- Add a new auth system using HMAC fingerprints.
- Create SignatureValidator utility class for secure signature operations
- Update job tokens to 256-bit
- Take care of time-replay attacks(3 hours)
- Added test for all corner cases.
- Update openapi specs.
I tested with this script:
```
set -e
BASE_URL="${X2A_BASE_URL:-http://localhost:7007/api/x2a}"
AUTH_URL="${AUTH_URL:-http://localhost:7007/api/auth}"
PROJECT_ID="$1"
PHASE="${2:-init}"
DB_PATH="${X2A_DB_PATH:-./packages/backend/x2a.sqlite}"
if ! command -v jq &> /dev/null; then
echo "Error: jq is required. Install with: sudo dnf install jq"
exit 1
fi
if ! command -v sqlite3 &> /dev/null; then
echo "Error: sqlite3 is required. Install with: sudo dnf install sqlite"
exit 1
fi
echo ""
echo "Creating job..."
echo "==============="
echo "Authenticating with guest token..."
AUTH_RESPONSE=$(curl -s -X GET "$AUTH_URL/guest/refresh" \
-H "Content-Type: application/json" 2>&1)
TOKEN=$(echo "$AUTH_RESPONSE" | jq -r '.backstageIdentity.token // empty')
if [ -z "$TOKEN" ]; then
echo "✗ Failed to get authentication token"
echo "Response: $AUTH_RESPONSE"
echo ""
echo "Is the backend running at $AUTH_URL?"
exit 1
fi
echo "✓ Authenticated"
if [ -z "$PROJECT_ID" ]; then
echo "Creating new project..."
PROJECT_RESPONSE=$(curl -s -X POST "$BASE_URL/projects" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"name": "Test Project '"$(date +%s)"'",
"description": "Test project",
"abbreviation": "TP'"$(date +%s | tail -c 4)"'",
"sourceRepoUrl": "https://github.com/test/source",
"targetRepoUrl": "https://github.com/test/target",
"sourceRepoBranch": "main",
"targetRepoBranch": "main"
}')
PROJECT_ID=$(echo "$PROJECT_RESPONSE" | jq -r '.id // empty')
if [ -z "$PROJECT_ID" ]; then
echo "✗ Failed to create project"
echo "Response: $PROJECT_RESPONSE"
exit 1
fi
echo "✓ Project created: $PROJECT_ID"
fi
echo "Creating $PHASE job..."
if [ "$PHASE" != "init" ]; then
echo "Creating module for $PHASE phase..."
MODULE_RESPONSE=$(curl -s -X POST "$BASE_URL/projects/$PROJECT_ID/modules" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"name": "Test Module",
"sourcePath": "/cookbooks/test"
}')
MODULE_ID=$(echo "$MODULE_RESPONSE" | jq -r '.id // empty')
if [ -z "$MODULE_ID" ]; then
echo "✗ Failed to create module"
echo "Response: $MODULE_RESPONSE"
exit 1
fi
echo "✓ Module created: $MODULE_ID"
# Module-level job
JOB_RESPONSE=$(curl -s -X POST "$BASE_URL/projects/$PROJECT_ID/modules/$MODULE_ID/run" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"phase": "'"$PHASE"'",
"sourceRepoAuth": {"token": "test-token"},
"targetRepoAuth": {"token": "test-token"}
}')
else
# Project-level init job (no phase parameter needed)
JOB_RESPONSE=$(curl -s -X POST "$BASE_URL/projects/$PROJECT_ID/run" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"sourceRepoAuth": {"token": "test-token"},
"targetRepoAuth": {"token": "test-token"}
}')
fi
JOB_ID=$(echo "$JOB_RESPONSE" | jq -r '.jobId // empty')
if [ -z "$JOB_ID" ] || [ "$JOB_ID" = "null" ]; then
echo "✗ Failed to create job"
echo "Response: $JOB_RESPONSE"
exit 1
fi
echo "✓ Job created: $JOB_ID"
echo "Fetching callback token from database..."
if [ ! -f "$DB_PATH" ]; then
echo "✗ Database file not found at: $DB_PATH"
echo ""
echo "Make sure the backend is configured to use a file-based SQLite database."
echo "Check app-config.yaml: backend.database.connection should be a file path."
exit 1
fi
CALLBACK_TOKEN=$(sqlite3 "$DB_PATH" "SELECT callback_token FROM jobs WHERE id = '$JOB_ID';" 2>&1)
if [ -z "$CALLBACK_TOKEN" ]; then
echo "✗ Failed to retrieve callback token"
echo ""
echo "Query the database manually:"
echo " sqlite3 $DB_PATH \"SELECT id, phase, callback_token FROM jobs WHERE id = '$JOB_ID';\""
exit 1
fi
echo "✓ Token retrieved"
echo ""
echo "=================================="
echo "Job Created Successfully!"
echo "=================================="
echo ""
echo "Project ID: $PROJECT_ID"
echo "Job ID: $JOB_ID"
echo "Phase: $PHASE"
echo "Callback Token: $CALLBACK_TOKEN"
echo ""
echo "Test with UV (X2Ansible convertor CLI):"
echo "----------------------------------------"
echo "uv run app.py report \\"
echo " --url \"http://localhost:7007/api/x2a/projects/$PROJECT_ID/collectArtifacts?phase=$PHASE\" \\"
echo " --job-id \"$JOB_ID\" \\"
echo " --callback-token \"$CALLBACK_TOKEN\" \\"
echo " --artifacts \"migration_plan:https://storage.example/plan.md\""
echo ""
echo "" | sed "s/JOB_ID/$JOB_ID/g; s/CALLBACK_TOKEN/$CALLBACK_TOKEN/g; s/PROJECT_ID/$PROJECT_ID/g; s/PHASE/$PHASE/g"
```
Fix FLPATH-3341
Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
* fix(x2a): fix PR comments
Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
* Add callback token on x2a-report
---------
Signed-off-by: Eloy Coto <eloy.coto@acalustra.com>
3 participants
User description
I tested with this script:
Fix FLPATH-3341
Hey, I just made a Pull Request!
✔️ Checklist
PR Type
Enhancement, Tests
Description
Implement HMAC-SHA256 signature validation for collectArtifacts endpoint
Add SignatureValidator utility class for secure signature operations
Upgrade callback tokens to 256-bit cryptographic strength
Add replay attack prevention with 3-hour job age validation
Refactor request handling with dedicated validation classes
Update OpenAPI specs with security scheme documentation
Diagram Walkthrough
File Walkthrough
5 files
Create HMAC-SHA256 signature validator utility classImplement HMAC signature validation and request handlingUpgrade callback token generation to 256-bitUpgrade callback token generation to 256-bitRestructure router to bypass OpenAPI parser for raw body access2 files
Add comprehensive tests for signature validationAdd signature validation test cases and update existing tests5 files
Document HMAC security scheme and replay attack preventionUpdate generated OpenAPI spec with security definitionsAdd X-Callback-Signature header to server API typeAdd X-Callback-Signature header to client API typeUpdate API report with signature header definition