Skip to content

CI: PoC for IaaS support in Praktika with MacOS smoke tests#95960

Merged
maxknv merged 7 commits intomasterfrom
ci_asg
Feb 12, 2026
Merged

CI: PoC for IaaS support in Praktika with MacOS smoke tests#95960
maxknv merged 7 commits intomasterfrom
ci_asg

Conversation

@maxknv
Copy link
Copy Markdown
Member

@maxknv maxknv commented Feb 4, 2026

Changelog category (leave one):

  • CI Fix or Improvement (changelog entry is not required)

This PR introduces Infrastructure as Code (IaaS) support in Praktika as a proof of concept, enabling automated deployment and configuration of CI infrastructure components.

Key changes:

  1. PoC for IaaS support in Praktika: Configure and deploy cloud CI infrastructure components (dedicated hosts, EC2 instances, ASGs, IAMProfile) via Python Praktika's objects

  2. Improved Praktika CLI interface: Deploy infrastructure with praktika infrastructure --deploy --only COMPONENTS

  3. Infrastructure configuration documentation: Added README about infrastructure configuration via Praktika

  4. MacOS infrastructure setup: Basic configuration for MacOS metal hosts and EC2 instances for macOS tests

  5. Smoke test: Added MacOS functional tests job to validate the infrastructure (deployed metal host, and ec2 ci runner)

This lays the groundwork for managing CI infrastructure declaratively through Praktika.

@maxknv maxknv marked this pull request as draft February 4, 2026 15:32
@maxknv maxknv self-assigned this Feb 4, 2026
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh Bot commented Feb 4, 2026

Workflow [PR], commit [c6e6ef0]

Summary:

@clickhouse-gh clickhouse-gh Bot added the pr-ci label Feb 4, 2026
@maxknv maxknv changed the title CI: Add MacOS tests CI: Add MacOS functional tests job Feb 4, 2026
@maxknv maxknv force-pushed the ci_asg branch 6 times, most recently from 2850ae0 to dc41c45 Compare February 6, 2026 17:35
maxknv and others added 4 commits February 9, 2026 11:29
When an affected job requires another job by name (not artifact name), treat it as an ordering-only dependency and exclude it from `all_required_artifacts`. This prevents keeping alive jobs that only serve non-artifact-producing dependencies.

Example: If Job D (affected) requires Job C (provides nothing), and Job C requires artifacts from Job B, Job B will now be correctly skipped.

Also added documentation for `native_jobs.py` explaining the job filtering logic.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@maxknv maxknv added the no-fast-tests Drops preliminary CI jobs: Style, clang-tidy, Fast Test label Feb 9, 2026
@maxknv maxknv force-pushed the ci_asg branch 3 times, most recently from 1c3a23d to 86f7d6a Compare February 12, 2026 15:56
@maxknv maxknv changed the title CI: Add MacOS functional tests job CI: PoC for IaaS support in Praktika with MacOS functional tests Feb 12, 2026
@maxknv maxknv changed the title CI: PoC for IaaS support in Praktika with MacOS functional tests CI: PoC for IaaS support in Praktika with MacOS smoke tests Feb 12, 2026
@maxknv maxknv removed the no-fast-tests Drops preliminary CI jobs: Style, clang-tidy, Fast Test label Feb 12, 2026
@maxknv maxknv force-pushed the ci_asg branch 2 times, most recently from 82dbf71 to e24ef5c Compare February 12, 2026 16:38
@maxknv maxknv marked this pull request as ready for review February 12, 2026 16:38
@maxknv maxknv requested a review from Copilot February 12, 2026 16:39
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a Praktika CI proof-of-concept for managing AWS-based CI infrastructure (Dedicated Hosts / EC2 / Image Builder / Launch Templates / ASGs) and adds a macOS smoke test job, along with CLI/Docs updates to deploy/shutdown infrastructure via praktika infrastructure.

Changes:

  • Add new infrastructure components (DedicatedHost, EC2Instance, IAMInstanceProfile, ImageBuilder, LaunchTemplate, AutoScalingGroup) and integrate them into CloudInfrastructure.
  • Extend Praktika CLI with an infrastructure subcommand supporting --deploy/--shutdown and --only.
  • Add macOS smoke test job and wire it into PR workflow dependencies; update workflow log timestamping.

Reviewed changes

Copilot reviewed 32 out of 42 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
ci/workflows/pull_request.py Adds macOS smoke test job dependency on the Darwin binary artifact.
ci/praktika/yaml_generator.py Changes job log timestamping implementation for generated workflows.
ci/praktika/native_jobs.py Adjusts affected-job filtering logic to distinguish artifact vs job-name requirements.
ci/praktika/native_jobs.md Documents Praktika “native jobs” and job filtering behavior.
ci/praktika/infrastructure/native/lambda_slack_worker.py Tweaks Slack message formatting (removes divider).
ci/praktika/infrastructure/native/README.md Updates infra deployment command docs to new CLI.
ci/praktika/infrastructure/launch_template.py Adds EC2 Launch Template IaC component.
ci/praktika/infrastructure/lambda_function.py Removes lambda config dump helper.
ci/praktika/infrastructure/image_builder.py Adds EC2 Image Builder IaC component.
ci/praktika/infrastructure/iam_instance_profile.py Adds IAM Role + Instance Profile IaC component.
ci/praktika/infrastructure/ec2_instance.py Adds EC2 instance IaC component (create/tag/start/terminate).
ci/praktika/infrastructure/dedicated_host.py Adds Dedicated Host pool IaC component (allocate/release).
ci/praktika/infrastructure/cloud.py Expands CloudInfrastructure to orchestrate multiple IaC components with --only support.
ci/praktika/infrastructure/autoscaling_group.py Adds Auto Scaling Group IaC component.
ci/praktika/infrastructure/README.md Updates infra module docs, adds examples for new components and --only/--shutdown.
ci/praktika/html_page_development.md Updates HTML upload invocation to the new infrastructure CLI.
ci/praktika/main.py Replaces old deploy/html subcommands with unified infrastructure command.
ci/jobs/smoke_test.py Adds macOS smoke test script that runs a basic server/query validation.
ci/jobs/fast_test.py Adjusts temp dir path and adds a Darwin-specific execution path.
ci/infra/scripts/user_data_macos.txt Adds macOS runner bootstrap user-data script (brew, packages, runner install, cloud-init).
ci/infra/cloud.py Adds PoC infra config for macOS hosts/instances and IAM instance profile.
ci/defs/job_configs.py Adds macOS smoke test job config; factors fast-test digest config.
ci/defs/defs.py Adds macOS self-hosted runner labels and macOS smoke test job name.
.github/workflows/vectorsearchstress.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/release_branches.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/nightly_statistics.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/nightly_jepsen.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/nightly_fuzzers.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/nightly_coverage.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/merge_queue.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/hourly.yml Updates log timestamping pipeline for Praktika-run steps.
.github/workflows/backport_branches.yml Updates log timestamping pipeline for Praktika-run steps.

Comment thread ci/praktika/yaml_generator.py Outdated
Comment thread ci/praktika/infrastructure/cloud.py
Comment thread ci/praktika/infrastructure/cloud.py
Comment thread ci/praktika/infrastructure/cloud.py Outdated
Comment thread ci/praktika/__main__.py Outdated
Comment thread ci/praktika/native_jobs.py
Comment thread ci/defs/defs.py Outdated
Comment thread ci/infra/cloud.py
@maxknv maxknv force-pushed the ci_asg branch 5 times, most recently from 8db9122 to 938a57a Compare February 12, 2026 19:23
@maxknv maxknv added this pull request to the merge queue Feb 12, 2026
Merged via the queue into master with commit 0a07dcd Feb 12, 2026
135 checks passed
@maxknv maxknv deleted the ci_asg branch February 12, 2026 23:03
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Feb 12, 2026
@nikitamikhaylov nikitamikhaylov added the post-approved Approved, but after the PR is merged. label Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

post-approved Approved, but after the PR is merged. pr-ci pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants