Skip to content

Commit d2720a0

Browse files
diberrydiberry
authored
Merge pull request #23 from Azure-Samples/copilot/validate-workflow-files
Add GitHub Actions workflow validation system
2 parents f4acf09 + dfbdd1a commit d2720a0

File tree

5 files changed

+7223
-0
lines changed

5 files changed

+7223
-0
lines changed

‎.github/workflows/README.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# GitHub Actions Workflows
2+
3+
This directory contains GitHub Actions workflows for the Azure TypeScript LangChainJS repository.
4+
5+
## Available Workflows
6+
7+
### 1. Validate Workflows (`validate-workflows.yml`)
8+
9+
This workflow automatically validates all GitHub Actions workflows in the repository to ensure they are properly configured and using up-to-date actions.
10+
11+
**Features:**
12+
- **Syntax Validation**: Uses `actionlint` to validate workflow YAML syntax and check for common issues
13+
- **Action Availability Check**: Verifies that all referenced GitHub Actions are still available
14+
- **Version Tracking**: Lists all actions and their versions in use
15+
- **Automated Reports**: Generates a detailed report of all actions and their status
16+
17+
**When it runs:**
18+
- On push/pull request when workflow files are modified
19+
- Weekly on Mondays at 9:00 AM UTC (scheduled check for outdated actions)
20+
- Manually via workflow dispatch
21+
22+
**Jobs:**
23+
1. **validate-workflows**: Runs actionlint to check syntax and best practices
24+
2. **detect-outdated-actions**: Extracts and analyzes all actions used in workflows, checks availability, and compares with latest versions
25+
3. **summary**: Provides a consolidated summary of all validation jobs
26+
27+
**Outputs:**
28+
- A downloadable report artifact containing all actions and versions in use
29+
- GitHub step summary with validation results
30+
31+
### 2. CI (`ci.yml`)
32+
33+
Continuous integration workflow that builds and tests the project.
34+
35+
**When it runs:**
36+
- On push to the `main` branch
37+
- On pull requests to the `main` branch
38+
- Manually via workflow dispatch
39+
40+
**Jobs:**
41+
1. **build**: Installs dependencies and builds the project
42+
43+
## Validation Tools
44+
45+
### actionlint
46+
A static checker for GitHub Actions workflow files. It catches common mistakes and enforces best practices.
47+
48+
**Installation:**
49+
```bash
50+
bash <(curl https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash)
51+
```
52+
53+
**Usage:**
54+
```bash
55+
actionlint .github/workflows/*.yml
56+
```
57+
58+
## Best Practices
59+
60+
1. **Pin Actions to Specific Versions**: Always use specific commit SHAs or version tags (e.g., `actions/checkout@v4`) instead of branches
61+
2. **Keep Actions Updated**: Regularly review and update actions to their latest versions
62+
3. **Test Workflow Changes**: Use workflow dispatch or create test branches to validate changes before merging
63+
4. **Review Weekly Reports**: Check the scheduled validation runs to identify outdated actions
64+
65+
## Adding New Workflows
66+
67+
When adding new workflows:
68+
69+
1. Create your workflow file in `.github/workflows/`
70+
2. Ensure it passes `actionlint` validation locally:
71+
```bash
72+
actionlint .github/workflows/your-new-workflow.yml
73+
```
74+
3. Commit and push - the validate-workflows workflow will automatically check your new file
75+
4. Review the validation results in the GitHub Actions tab
76+
77+
## Troubleshooting
78+
79+
### Workflow Validation Failures
80+
81+
If the validate-workflows workflow fails:
82+
83+
1. Check the job logs for specific error messages
84+
2. Run `actionlint` locally to get detailed error information
85+
3. Fix the issues and push the changes
86+
4. The validation will run automatically on your next push
87+
88+
### Common Issues
89+
90+
- **Syntax Errors**: Check YAML indentation and syntax
91+
- **Deprecated Actions**: Update to the latest version of the action
92+
- **Unknown Actions**: Verify the action name and that the repository is public
93+
- **Shell Script Issues**: Ensure proper quoting and use of variables in shell scripts
94+
95+
## Security Considerations
96+
97+
- Only use actions from trusted sources (verified creators or well-known organizations)
98+
- Pin actions to specific commit SHAs for maximum security
99+
- Review action source code before using new actions
100+
- Keep actions updated to receive security patches
101+
102+
## Resources
103+
104+
- [GitHub Actions Documentation](https://docs.github.com/en/actions)
105+
- [actionlint Repository](https://github.com/rhysd/actionlint)
106+
- [GitHub Actions Security Best Practices](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions)

‎.github/workflows/ci.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
workflow_dispatch:
9+
10+
permissions:
11+
contents: read
12+
13+
jobs:
14+
build:
15+
name: Build and Test
16+
runs-on: ubuntu-latest
17+
permissions:
18+
contents: read
19+
20+
steps:
21+
- name: Checkout repository
22+
uses: actions/checkout@v4
23+
24+
- name: Setup Node.js
25+
uses: actions/setup-node@v4
26+
with:
27+
node-version: '20'
28+
29+
- name: Install dependencies
30+
run: npm install
31+
32+
- name: Build project
33+
run: npm run build

‎.github/workflows/validate-workflows.yml‎

Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
name: Validate GitHub Actions Workflows
2+
3+
on:
4+
push:
5+
paths:
6+
- '.github/workflows/**'
7+
pull_request:
8+
paths:
9+
- '.github/workflows/**'
10+
workflow_dispatch:
11+
schedule:
12+
# Run weekly on Monday at 9:00 AM UTC to check for outdated actions
13+
- cron: '0 9 * * 1'
14+
15+
permissions:
16+
contents: read
17+
18+
jobs:
19+
validate-workflows:
20+
name: Validate Workflow Files
21+
runs-on: ubuntu-latest
22+
permissions:
23+
contents: read
24+
25+
steps:
26+
- name: Checkout repository
27+
uses: actions/checkout@v4
28+
29+
- name: Install actionlint
30+
run: |
31+
# Download actionlint v1.7.9 (pinned version with checksum verification)
32+
ACTIONLINT_VERSION="1.7.9"
33+
ACTIONLINT_URL="https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz"
34+
EXPECTED_SHA256="233b280d05e100837f4af1433c7b40a5dcb306e3aa68fb4f17f8a7f45a7df7b4"
35+
36+
curl -sL "$ACTIONLINT_URL" -o actionlint.tar.gz
37+
echo "$EXPECTED_SHA256 actionlint.tar.gz" | sha256sum -c -
38+
tar xzf actionlint.tar.gz
39+
sudo mv ./actionlint /usr/local/bin/
40+
rm actionlint.tar.gz
41+
actionlint --version
42+
43+
- name: Run actionlint
44+
run: |
45+
echo "Running actionlint to validate workflow syntax and best practices..."
46+
actionlint -color
47+
continue-on-error: false
48+
49+
detect-outdated-actions:
50+
name: Detect Outdated Actions
51+
runs-on: ubuntu-latest
52+
permissions:
53+
contents: read
54+
55+
steps:
56+
- name: Checkout repository
57+
uses: actions/checkout@v4
58+
59+
- name: Extract actions from workflows
60+
id: extract-actions
61+
run: |
62+
echo "Extracting actions from workflow files..."
63+
mkdir -p /tmp/workflow-analysis
64+
65+
# Extract all 'uses:' lines from workflow files
66+
find .github/workflows -name "*.yml" -o -name "*.yaml" | while IFS= read -r workflow; do
67+
echo "Analyzing $workflow"
68+
grep -E "^\s*uses:" "$workflow" | sed 's/.*uses:\s*//' | sed 's/#.*//' | sed 's/\s*$//' >> /tmp/workflow-analysis/actions.txt || true
69+
done
70+
71+
# Remove duplicates and sort
72+
if [ -f /tmp/workflow-analysis/actions.txt ]; then
73+
sort -u /tmp/workflow-analysis/actions.txt > /tmp/workflow-analysis/unique_actions.txt
74+
echo "Found actions:"
75+
cat /tmp/workflow-analysis/unique_actions.txt
76+
77+
# Count actions
78+
ACTION_COUNT=$(wc -l < /tmp/workflow-analysis/unique_actions.txt)
79+
echo "Total unique actions: $ACTION_COUNT"
80+
else
81+
echo "No actions found in workflows"
82+
fi
83+
84+
- name: Install jq for JSON parsing
85+
run: sudo apt-get update && sudo apt-get install -y jq
86+
87+
- name: Check action availability and versions
88+
env:
89+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
90+
run: |
91+
echo "Checking if actions are available on GitHub..."
92+
93+
if [ ! -f /tmp/workflow-analysis/unique_actions.txt ]; then
94+
echo "No actions to check"
95+
exit 0
96+
fi
97+
98+
UNAVAILABLE_ACTIONS=""
99+
100+
while IFS= read -r action; do
101+
# Skip local actions (starting with ./)
102+
if [[ "$action" == ./* ]]; then
103+
echo "✓ Local action: $action (skipped)"
104+
continue
105+
fi
106+
107+
# Check if action has a version specified
108+
if [[ "$action" != *"@"* ]]; then
109+
echo "⚠️ Warning: Action without version: $action"
110+
continue
111+
fi
112+
113+
# Extract owner/repo and version
114+
ACTION_PATH=$(echo "$action" | cut -d'@' -f1)
115+
ACTION_VERSION=$(echo "$action" | cut -d'@' -f2)
116+
117+
# Skip if no valid path
118+
if [[ "$ACTION_PATH" != *"/"* ]]; then
119+
continue
120+
fi
121+
122+
echo "Checking $ACTION_PATH@$ACTION_VERSION..."
123+
124+
# Use GitHub API with authentication for better rate limits
125+
API_URL="https://api.github.com/repos/$ACTION_PATH"
126+
RESPONSE=$(curl -s -w "\n%{http_code}" \
127+
-H "Authorization: token $GITHUB_TOKEN" \
128+
-H "Accept: application/vnd.github+json" \
129+
-H "X-GitHub-Api-Version: 2022-11-28" \
130+
"$API_URL")
131+
132+
HTTP_CODE=$(echo "$RESPONSE" | tail -n1)
133+
134+
if [ "$HTTP_CODE" -eq 200 ]; then
135+
echo "✓ Action available: $action"
136+
137+
# Try to fetch latest release for comparison
138+
RELEASE_RESPONSE=$(curl -s \
139+
-H "Authorization: token $GITHUB_TOKEN" \
140+
-H "Accept: application/vnd.github+json" \
141+
-H "X-GitHub-Api-Version: 2022-11-28" \
142+
"$API_URL/releases/latest")
143+
144+
LATEST_RELEASE=$(echo "$RELEASE_RESPONSE" | jq -r '.tag_name // empty')
145+
146+
if [ -n "$LATEST_RELEASE" ] && [ "$ACTION_VERSION" != "$LATEST_RELEASE" ]; then
147+
echo " ℹ️ Latest version available: $LATEST_RELEASE (current: $ACTION_VERSION)"
148+
fi
149+
else
150+
echo "✗ Action not found or inaccessible: $action (HTTP $HTTP_CODE)"
151+
UNAVAILABLE_ACTIONS="${UNAVAILABLE_ACTIONS}${action}"$'\n'
152+
fi
153+
done < /tmp/workflow-analysis/unique_actions.txt
154+
155+
if [ -n "$UNAVAILABLE_ACTIONS" ]; then
156+
echo ""
157+
echo "⚠️ Warning: Some actions are unavailable:"
158+
echo "$UNAVAILABLE_ACTIONS"
159+
fi
160+
161+
- name: Generate action version report
162+
run: |
163+
{
164+
echo "## Action Version Report"
165+
echo ""
166+
echo "This report lists all GitHub Actions used in workflows:"
167+
echo ""
168+
} > /tmp/workflow-analysis/report.md
169+
170+
if [ -f /tmp/workflow-analysis/unique_actions.txt ]; then
171+
{
172+
echo "| Action | Version | Status |"
173+
echo "|--------|---------|--------|"
174+
} >> /tmp/workflow-analysis/report.md
175+
176+
while IFS= read -r action; do
177+
ACTION_PATH=$(echo "$action" | cut -d'@' -f1)
178+
ACTION_VERSION=$(echo "$action" | cut -d'@' -f2)
179+
echo "| \`$ACTION_PATH\` | \`$ACTION_VERSION\` | ✓ In use |" >> /tmp/workflow-analysis/report.md
180+
done < /tmp/workflow-analysis/unique_actions.txt
181+
else
182+
echo "No actions found in workflows." >> /tmp/workflow-analysis/report.md
183+
fi
184+
185+
{
186+
echo ""
187+
echo "---"
188+
echo "Generated on: $(date -u)"
189+
} >> /tmp/workflow-analysis/report.md
190+
191+
cat /tmp/workflow-analysis/report.md
192+
193+
- name: Upload report
194+
uses: actions/upload-artifact@v4
195+
if: always()
196+
with:
197+
name: workflow-validation-report
198+
path: /tmp/workflow-analysis/report.md
199+
retention-days: 30
200+
201+
summary:
202+
name: Validation Summary
203+
runs-on: ubuntu-latest
204+
needs: [validate-workflows, detect-outdated-actions]
205+
if: always()
206+
permissions:
207+
contents: read
208+
209+
steps:
210+
- name: Create summary
211+
run: |
212+
{
213+
echo "# Workflow Validation Summary"
214+
echo ""
215+
echo "✅ All workflow validation jobs completed"
216+
echo ""
217+
echo "## Jobs Status:"
218+
echo "- Validate Workflows: ${{ needs.validate-workflows.result }}"
219+
echo "- Detect Outdated Actions: ${{ needs.detect-outdated-actions.result }}"
220+
} >> "$GITHUB_STEP_SUMMARY"

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,3 +146,6 @@ build/
146146
# Bicep compiled ARM templates
147147
infra/**/*.json
148148
!infra/**/*.parameters.json
149+
150+
# actionlint binary (downloaded for local testing)
151+
actionlint

0 commit comments

Comments
 (0)