-
Notifications
You must be signed in to change notification settings - Fork 513
emu with a q stuff #1266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
emu with a q stuff #1266
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
f8fcf9d
Squashed commit of the following:
mantrakp04 0010642
Refactor QEMU emulator setup and consolidate cloud-init configurations
mantrakp04 c2e4698
Refactor local emulator configuration management and remove obsolete …
mantrakp04 4508f24
Enhance Dockerfile for local emulator and server setup
mantrakp04 dec62a0
Refactor QEMU emulator build and snapshot handling
mantrakp04 580ff10
Refactor local emulator setup and consolidate services
mantrakp04 e3592bb
Add base image fingerprinting for QEMU snapshots
mantrakp04 5a729ec
Enhance configuration and permissions in local emulator setup
mantrakp04 c1a15ff
Enhance local emulator configuration and file management
mantrakp04 0496cd2
Add QEMU emulator build workflow and emulator command integration
mantrakp04 4a43a88
Update emulator configuration and enhance CLI functionality
mantrakp04 fe16a71
Merge branch 'dev' into emu-with-a-q
mantrakp04 5d98b44
Remove deprecated configuration files and streamline emulator CLI com…
mantrakp04 3321e5c
Update QEMU emulator scripts to improve KVM detection and service sta…
mantrakp04 8336d48
Refactor local emulator type handling and architecture detection
mantrakp04 298062a
Enhance local emulator configuration handling and file management
mantrakp04 578593a
Enhance QEMU emulator build workflow and local emulator configuration
mantrakp04 80ee1ab
Merge branch 'dev' into emu-with-a-q
mantrakp04 42bc125
Enhance setBranchConfigOverride for local emulator file handling
mantrakp04 4be8a82
Update QEMU emulator build workflow for improved performance and reli…
mantrakp04 f3514f8
Refactor QEMU emulator build and local emulator configuration
mantrakp04 d5d2a09
Increase EMULATOR_READY_TIMEOUT in QEMU emulator build workflow
mantrakp04 e509947
Enhance local emulator setup with environment generation and configur…
mantrakp04 ad2db19
Refactor local emulator configuration validation and enhance CLI tests
mantrakp04 a233e64
Update local emulator environment handling and documentation
mantrakp04 14f8164
Enhance local emulator project handling and configuration validation
mantrakp04 9d0f7c1
Refactor local emulator scripts and configuration management
mantrakp04 77d87fd
Refactor local emulator setup and enhance GitHub Actions workflow
mantrakp04 bef4aab
Enhance QEMU emulator build workflow and add documentation
mantrakp04 6d9d4e7
Merge branch 'dev' into emu-with-a-q
mantrakp04 771ed15
Update CLI tests for emulator commands
mantrakp04 0f3daeb
Merge branch 'dev' into emu-with-a-q
mantrakp04 950d74a
remove stack.config.ts
mantrakp04 26254c4
Merge branch 'dev' into emu-with-a-q
mantrakp04 7daea68
Increase QEMU smoke test timeout to 60 minutes
mantrakp04 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,255 @@ | ||
| name: Build & Publish QEMU Emulator Images | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| - dev | ||
| pull_request: | ||
| paths: | ||
| - 'docker/local-emulator/**' | ||
| - '.github/workflows/qemu-emulator-build.yaml' | ||
| workflow_dispatch: | ||
| inputs: | ||
| publish: | ||
| description: 'Publish images to GitHub Releases' | ||
| type: boolean | ||
| default: false | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/dev' }} | ||
|
|
||
| env: | ||
| EMULATOR_IMAGE_NAME: stack-local-emulator | ||
|
|
||
| jobs: | ||
| build: | ||
| name: Build QEMU Image (${{ matrix.arch }}) | ||
| runs-on: ubicloud-standard-8 | ||
| timeout-minutes: 120 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - arch: amd64 | ||
| - arch: arm64 | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Set up QEMU user-mode emulation | ||
| uses: docker/setup-qemu-action@v3 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Install QEMU dependencies | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get install -y qemu-system-x86 qemu-system-arm qemu-utils genisoimage socat qemu-efi-aarch64 | ||
|
|
||
| - name: Build QEMU image | ||
| run: | | ||
| chmod +x docker/local-emulator/qemu/build-image.sh | ||
| EMULATOR_PROVISION_TIMEOUT=6000 \ | ||
| docker/local-emulator/qemu/build-image.sh ${{ matrix.arch }} | ||
|
|
||
| - name: Generate emulator env | ||
| run: node docker/local-emulator/generate-env-development.mjs | ||
|
|
||
| - name: Start emulator and verify | ||
| run: | | ||
| chmod +x docker/local-emulator/qemu/run-emulator.sh | ||
| EMULATOR_ARCH=${{ matrix.arch }} \ | ||
| EMULATOR_READY_TIMEOUT=3200 \ | ||
| docker/local-emulator/qemu/run-emulator.sh start | ||
|
|
||
| - name: Verify services are healthy | ||
| run: | | ||
| EMULATOR_ARCH=${{ matrix.arch }} \ | ||
| docker/local-emulator/qemu/run-emulator.sh status | ||
|
|
||
| - name: Stop emulator | ||
| if: always() | ||
| run: | | ||
| EMULATOR_ARCH=${{ matrix.arch }} \ | ||
| docker/local-emulator/qemu/run-emulator.sh stop | ||
|
mantrakp04 marked this conversation as resolved.
|
||
|
|
||
| - name: Package image | ||
| run: | | ||
| BASE_IMG="docker/local-emulator/qemu/images/stack-emulator-${{ matrix.arch }}.qcow2" | ||
| cp "$BASE_IMG" "stack-emulator-${{ matrix.arch }}.qcow2" | ||
|
|
||
| - name: Upload image artifact | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: qemu-emulator-${{ matrix.arch }} | ||
| path: stack-emulator-${{ matrix.arch }}.qcow2 | ||
| retention-days: 30 | ||
| compression-level: 0 | ||
|
|
||
| test: | ||
| name: Smoke Test (${{ matrix.arch }}) | ||
| needs: build | ||
| runs-on: ubicloud-standard-8 | ||
| timeout-minutes: 60 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - arch: amd64 | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Install QEMU dependencies | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get install -y qemu-system-x86 qemu-utils genisoimage socat | ||
|
|
||
| - name: Download built image | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: qemu-emulator-${{ matrix.arch }} | ||
| path: docker/local-emulator/qemu/images/ | ||
|
|
||
| - name: Generate emulator env | ||
| run: node docker/local-emulator/generate-env-development.mjs | ||
|
|
||
| - name: Start emulator from artifact | ||
| run: | | ||
| chmod +x docker/local-emulator/qemu/run-emulator.sh docker/local-emulator/qemu/common.sh | ||
| EMULATOR_ARCH=${{ matrix.arch }} \ | ||
| EMULATOR_READY_TIMEOUT=600 \ | ||
| docker/local-emulator/qemu/run-emulator.sh start | ||
|
|
||
| - name: Verify services are healthy | ||
| run: | | ||
| EMULATOR_ARCH=${{ matrix.arch }} \ | ||
| docker/local-emulator/qemu/run-emulator.sh status | ||
|
|
||
| - name: Smoke test — backend health | ||
| run: curl -sf http://localhost:26701/health?db=1 | ||
|
|
||
| - name: Smoke test — dashboard reachable | ||
| run: curl -sf -o /dev/null -w "HTTP %{http_code}\n" http://localhost:26700/handler/sign-in | ||
|
|
||
| - name: Smoke test — MinIO health | ||
| run: curl -sf http://localhost:26702/minio/health/live | ||
|
|
||
| - name: Smoke test — Inbucket reachable | ||
| run: curl -sf -o /dev/null -w "HTTP %{http_code}\n" http://localhost:26703/ | ||
|
|
||
| - name: Stop emulator | ||
| if: always() | ||
| run: | | ||
| EMULATOR_ARCH=${{ matrix.arch }} \ | ||
| docker/local-emulator/qemu/run-emulator.sh stop | ||
|
|
||
| - name: Print serial log on failure | ||
| if: failure() | ||
| run: tail -100 docker/local-emulator/qemu/run/vm/serial.log 2>/dev/null || true | ||
|
|
||
| publish: | ||
| name: Publish to GitHub Releases | ||
| needs: [build, test] | ||
| if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev' || (github.event_name == 'workflow_dispatch' && inputs.publish) | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Download all artifacts | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| path: artifacts | ||
|
|
||
| - name: Prepare release assets | ||
| run: | | ||
| mkdir -p release | ||
| SHORT_SHA="${GITHUB_SHA:0:8}" | ||
| BRANCH="${GITHUB_REF_NAME}" | ||
| DATE="$(date -u +%Y%m%d)" | ||
| TAG="emulator-${BRANCH}-${DATE}-${SHORT_SHA}" | ||
| echo "RELEASE_TAG=${TAG}" >> "$GITHUB_ENV" | ||
| echo "SHORT_SHA=${SHORT_SHA}" >> "$GITHUB_ENV" | ||
|
|
||
| for f in artifacts/qemu-emulator-*/*.qcow2; do | ||
| cp "$f" release/ | ||
| done | ||
|
|
||
| cat > release-notes.md <<EOF | ||
| ## QEMU Emulator Images | ||
|
|
||
| Built from \`${BRANCH}\` @ \`${GITHUB_SHA}\` | ||
|
|
||
| ### Images | ||
| | File | Description | | ||
| |------|-------------| | ||
| | \`stack-emulator-arm64.qcow2\` | ARM64 emulator image | | ||
| | \`stack-emulator-amd64.qcow2\` | AMD64 emulator image | | ||
|
|
||
| ### Usage | ||
| \`\`\`bash | ||
| stack emulator pull | ||
| stack emulator start | ||
| \`\`\` | ||
| EOF | ||
|
|
||
| ls -lh release/ | ||
|
|
||
| - name: Create or update GitHub Release | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| TITLE="QEMU Emulator — ${{ github.ref_name }} ($SHORT_SHA)" | ||
|
|
||
| if gh release view "$RELEASE_TAG" >/dev/null 2>&1; then | ||
| gh release edit "$RELEASE_TAG" \ | ||
| --title "$TITLE" \ | ||
| --notes-file release-notes.md \ | ||
| --prerelease | ||
| gh release upload "$RELEASE_TAG" release/* --clobber | ||
| else | ||
| gh release create "$RELEASE_TAG" \ | ||
| --title "$TITLE" \ | ||
| --notes-file release-notes.md \ | ||
| --prerelease \ | ||
| release/* | ||
| fi | ||
|
|
||
| - name: Update latest tag for branch | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| LATEST_TAG="emulator-${{ github.ref_name }}-latest" | ||
| TITLE="QEMU Emulator — ${{ github.ref_name }} (latest)" | ||
| NOTES="Latest emulator images from \`${{ github.ref_name }}\`. Auto-updated on each build." | ||
|
|
||
| if gh release view "$LATEST_TAG" >/dev/null 2>&1; then | ||
| gh release edit "$LATEST_TAG" \ | ||
| --draft \ | ||
| --prerelease \ | ||
| --target "${{ github.sha }}" \ | ||
| --title "$TITLE" \ | ||
| --notes "$NOTES" | ||
| else | ||
| gh release create "$LATEST_TAG" \ | ||
| --draft \ | ||
| --prerelease \ | ||
| --target "${{ github.sha }}" \ | ||
| --title "$TITLE" \ | ||
| --notes "$NOTES" \ | ||
| || gh release edit "$LATEST_TAG" \ | ||
| --draft \ | ||
| --prerelease \ | ||
| --target "${{ github.sha }}" \ | ||
| --title "$TITLE" \ | ||
| --notes "$NOTES" | ||
| fi | ||
|
|
||
| gh release upload "$LATEST_TAG" release/* --clobber | ||
| gh release edit "$LATEST_TAG" --draft=false --prerelease | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.