Skip to content

example of starting ax experiment from dataframe data (#5098)#5098

Closed
sdaulton wants to merge 1 commit into
facebook:mainfrom
sdaulton:export-D93760064
Closed

example of starting ax experiment from dataframe data (#5098)#5098
sdaulton wants to merge 1 commit into
facebook:mainfrom
sdaulton:export-D93760064

Conversation

@sdaulton
Copy link
Copy Markdown
Contributor

@sdaulton sdaulton commented Mar 25, 2026

Summary:

This diff adds a new Ax tutorial notebook demonstrating how to initialize an experiment from pre-existing DataFrame data.

Key Features Covered:

  1. Attaching Historical Trials — Shows how to use client.attach_trial() and client.complete_trial() to import historical experiment data (arms with parameters and their observed metric values) from a pandas DataFrame
  2. Warm-Starting Bayesian Optimization — After attaching historical data, uses Ax's Client to generate new candidate trials with model-based optimization (BoTorch)
  3. Analysis & Visualization — Demonstrates Ax's built-in analysis tools: cross-validation plots, utility progression tracking, and arm effects visualization (both observed and predicted)

Use Cases:

  • Migrating experiments run outside of Ax into the Ax framework
  • Leveraging existing CSV/database data to warm-start Bayesian optimization
  • Building on historical configurations and outcomes

Technical Details:

  • Uses the Branin function as a benchmark optimization problem (2 parameters, known global minimum ≈ 0.398)
  • Creates 15 quasi-random historical evaluations and attaches them as completed trials
  • Generates and evaluates a new candidate using client.get_next_trial()
  • Compares predicted vs. observed arm effects

Reviewed By: andycylmeta

Differential Revision: D93760064

@meta-cla meta-cla Bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Mar 25, 2026
@meta-codesync
Copy link
Copy Markdown

meta-codesync Bot commented Mar 25, 2026

@sdaulton has exported this pull request. If you are a Meta employee, you can view the originating Diff in D93760064.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.41%. Comparing base (218bf55) to head (5fe7341).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5098   +/-   ##
=======================================
  Coverage   96.41%   96.41%           
=======================================
  Files         613      613           
  Lines       68131    68131           
=======================================
  Hits        65687    65687           
  Misses       2444     2444           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@meta-codesync meta-codesync Bot changed the title example of starting ax experiment from dataframe data example of starting ax experiment from dataframe data (#5098) Mar 30, 2026
sdaulton added a commit to sdaulton/Ax-1 that referenced this pull request Mar 30, 2026
Summary:

This diff adds a new Ax tutorial notebook demonstrating how to initialize an experiment from pre-existing DataFrame data.

**Key Features Covered:**
1. **Attaching Historical Trials** — Shows how to use `client.attach_trial()` and `client.complete_trial()` to import historical experiment data (arms with parameters and their observed metric values) from a pandas DataFrame
2. **Warm-Starting Bayesian Optimization** — After attaching historical data, uses Ax's `Client` to generate new candidate trials with model-based optimization (BoTorch)
3. **Analysis & Visualization** — Demonstrates Ax's built-in analysis tools: cross-validation plots, utility progression tracking, and arm effects visualization (both observed and predicted)

**Use Cases:**
- Migrating experiments run outside of Ax into the Ax framework
- Leveraging existing CSV/database data to warm-start Bayesian optimization
- Building on historical configurations and outcomes

**Technical Details:**
- Uses the Branin function as a benchmark optimization problem (2 parameters, known global minimum ≈ 0.398)
- Creates 15 quasi-random historical evaluations and attaches them as completed trials
- Generates and evaluates a new candidate using `client.get_next_trial()`
- Compares predicted vs. observed arm effects

Reviewed By: andycylmeta

Differential Revision: D93760064
sdaulton added a commit to sdaulton/Ax-1 that referenced this pull request Mar 30, 2026
Summary:

This diff adds a new Ax tutorial notebook demonstrating how to initialize an experiment from pre-existing DataFrame data.

**Key Features Covered:**
1. **Attaching Historical Trials** — Shows how to use `client.attach_trial()` and `client.complete_trial()` to import historical experiment data (arms with parameters and their observed metric values) from a pandas DataFrame
2. **Warm-Starting Bayesian Optimization** — After attaching historical data, uses Ax's `Client` to generate new candidate trials with model-based optimization (BoTorch)
3. **Analysis & Visualization** — Demonstrates Ax's built-in analysis tools: cross-validation plots, utility progression tracking, and arm effects visualization (both observed and predicted)

**Use Cases:**
- Migrating experiments run outside of Ax into the Ax framework
- Leveraging existing CSV/database data to warm-start Bayesian optimization
- Building on historical configurations and outcomes

**Technical Details:**
- Uses the Branin function as a benchmark optimization problem (2 parameters, known global minimum ≈ 0.398)
- Creates 15 quasi-random historical evaluations and attaches them as completed trials
- Generates and evaluates a new candidate using `client.get_next_trial()`
- Compares predicted vs. observed arm effects

Reviewed By: andycylmeta

Differential Revision: D93760064
Summary:
Pull Request resolved: facebook#5098

This diff adds a new Ax tutorial notebook demonstrating how to initialize an experiment from pre-existing DataFrame data.

**Key Features Covered:**
1. **Attaching Historical Trials** — Shows how to use `client.attach_trial()` and `client.complete_trial()` to import historical experiment data (arms with parameters and their observed metric values) from a pandas DataFrame
2. **Warm-Starting Bayesian Optimization** — After attaching historical data, uses Ax's `Client` to generate new candidate trials with model-based optimization (BoTorch)
3. **Analysis & Visualization** — Demonstrates Ax's built-in analysis tools: cross-validation plots, utility progression tracking, and arm effects visualization (both observed and predicted)

**Use Cases:**
- Migrating experiments run outside of Ax into the Ax framework
- Leveraging existing CSV/database data to warm-start Bayesian optimization
- Building on historical configurations and outcomes

**Technical Details:**
- Uses the Branin function as a benchmark optimization problem (2 parameters, known global minimum ≈ 0.398)
- Creates 15 quasi-random historical evaluations and attaches them as completed trials
- Generates and evaluates a new candidate using `client.get_next_trial()`
- Compares predicted vs. observed arm effects

Reviewed By: andycylmeta

Differential Revision: D93760064
@meta-codesync
Copy link
Copy Markdown

meta-codesync Bot commented Mar 30, 2026

This pull request has been merged in 7c67537.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported Merged meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants