Skip to content

Migrate CI to protected runners and JFrog PyPI proxy#770

Merged
vikrantpuppala merged 8 commits intomainfrom
ci/protected-runners-jfrog
Apr 13, 2026
Merged

Migrate CI to protected runners and JFrog PyPI proxy#770
vikrantpuppala merged 8 commits intomainfrom
ci/protected-runners-jfrog

Conversation

@vikrantpuppala
Copy link
Copy Markdown
Contributor

@vikrantpuppala vikrantpuppala commented Apr 13, 2026

Summary

  • Switched all workflow jobs (9 total across 5 workflows) from ubuntu-latest to databricks-protected-runner-group with linux-ubuntu-latest labels
  • Added .github/actions/setup-jfrog/action.yml composite action that obtains a JFrog access token via GitHub OIDC and sets PIP_INDEX_URL so all pip/poetry installs route through databricks.jfrog.io/artifactory/api/pypi/db-pypi
  • Added id-token: write permission to all workflows that install Python packages (required for the OIDC token exchange)
  • Setup mirrors the pattern used in databricks-odbc for consistency across repos

Test plan

  • Verify all CI checks pass on this PR (protected runner access + JFrog OIDC working)
  • Confirm package installation succeeds through JFrog proxy (check poetry install logs)
  • DCO check still works (no JFrog needed, just runner change)

This pull request was AI-assisted by Isaac.

tejassp-db
tejassp-db approved these changes Apr 13, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@tejassp-db tejassp-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude generated code with security hardening doc looks good.

@vikrantpuppala
Migrate CI to databricks-protected runners and route PyPI through JFrog
2bf65b9 
Protected runners are required for Databricks OSS repos. Add a
setup-jfrog composite action (OIDC-based, matching databricks-odbc) that
sets PIP_INDEX_URL so all pip/poetry installs go through the JFrog PyPI
proxy. Every workflow now runs on the databricks-protected-runner-group
with id-token: write for the OIDC exchange.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala
Add Poetry JFrog source configuration to all workflows
96f33ce 
The previous commit only set PIP_INDEX_URL, but Poetry uses its own
resolver and needs explicit source configuration. Add a
"Configure Poetry for JFrog" step after poetry install in every job
that sets up the JFrog repository and credentials, then adds it as
the primary source for the project.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala
Fix step ordering: move JFrog setup after poetry install
c0bbbcf 
The snok/install-poetry action uses pip internally to install poetry.
When PIP_INDEX_URL was set before this step, the installer tried to
route through JFrog and failed with an SSL error. Move the JFrog OIDC
token + PIP_INDEX_URL + poetry source configuration to run after
Install Poetry but before poetry install.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala
Replace snok/install-poetry with pip install through JFrog
abca166 
The hardened runners block direct access to install.python-poetry.org,
causing snok/install-poetry to fail with SSL errors. Replace it with
`pip install poetry==2.2.1` which routes through the JFrog PyPI proxy.

New step ordering: checkout → setup-python → Setup JFrog (OIDC +
PIP_INDEX_URL) → pip install poetry → Configure Poetry for JFrog →
poetry install.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala
Add poetry lock --no-update after source add to fix lock mismatch
7a5740d 
poetry source add modifies pyproject.toml, which makes poetry refuse
to install from the existing lock file. Running poetry lock --no-update
regenerates the lock file metadata without changing dependency versions.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala
Fix poetry lock flag and YAML indentation
2067a8a 
Poetry 2.x doesn't have --no-update flag, use poetry lock instead.
Also fix indentation of poetry lock in the arrow test job.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala
Move JFrog setup before setup-python, matching sqlalchemy pattern
002b492 
Follow the proven pattern from databricks/databricks-sqlalchemy#59:
checkout → Setup JFrog → setup-python → pip install poetry → poetry
source add + poetry lock → poetry install.

The hardened runners block pypi.org at the network level, so JFrog
must be configured before actions/setup-python (which upgrades pip).
Also simplified workflows by removing verbose section comments.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala
Extract setup-poetry composite action to remove duplication
23489a6 
Create .github/actions/setup-poetry that bundles JFrog setup,
setup-python, poetry install via pip, JFrog source config, cache,
and dependency install into a single reusable action with inputs
for python-version, install-args, cache-path, and cache-suffix.

All workflows now call setup-poetry instead of repeating these steps,
matching the pattern from databricks/databricks-sqlalchemy#59.

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
@vikrantpuppala vikrantpuppala force-pushed the ci/protected-runners-jfrog branch from 13430c3 to 23489a6 Compare April 13, 2026 12:04
@vikrantpuppala vikrantpuppala merged commit fbdcd32 into main Apr 13, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tejassp-db tejassp-db tejassp-db approved these changes

@jprakash-db jprakash-db Awaiting requested review from jprakash-db

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vikrantpuppala @tejassp-db