Pattern: run GX Core in ephemeral cloud sandboxes (EphemeralDataContext) + pan-provider example

[zozo123]

GX Core's ephemeral Data Context (gx.get_context(mode="ephemeral")) is a natural fit for throwaway/cloud sandbox environments — disposable containers, hosted notebooks, and CI runners — because nothing is written to a persistent metadata store. Sharing a small reproducible pattern in case it's useful to others, and to ask whether a short docs note would be welcome.

Minimal pattern (pandas, in-memory)

import great_expectations as gx
import pandas as pd

context = gx.get_context(mode="ephemeral")  # nothing persists between runs
df = pd.DataFrame({"passenger_count": [1, 2, 3, 4]})

batch = (
    context.data_sources.add_pandas("pandas")
    .add_dataframe_asset("trips")
    .add_batch_definition_whole_dataframe("batch")
    .get_batch(batch_parameters={"dataframe": df})
)
result = batch.validate(
    gx.expectations.ExpectColumnValuesToBeBetween(column="passenger_count", min_value=1, max_value=6)
)
print(result.success)

Install fresh each run (pip install great_expectations), validate, and discard the environment on exit — no gx/ project directory committed, no persistent backend.

Running it across sandbox providers

Because GX itself is runtime-agnostic, the same script runs unchanged in any sandbox. I put together a small open-source demo with provider-agnostic runners behind one run(script) interface — verified end-to-end on islo.dev (cloud Firecracker microVM) and a local Apple container, with Daytona implemented and Modal/E2B planned:

https://github.com/zozo123/gx-sandbox-demo

(Disclosure: that's my own demo repo — sharing it as a worked example, not a product.)

Question

Would a short, provider-agnostic note on the Create a Data Context (Ephemeral) page — covering ephemeral mode for CI / disposable compute / no-persistent-filesystem use — be a welcome [DOCS] contribution? Happy to open a small PR if so. Also curious which other sandbox environments people run GX Core in.

[joshua-stauffer]

hey @zozo123 , thanks for sharing! Feel free to open a docs PR. The code snippet on the EphemeralDataContext setup page should remain DataContext focused, but you're right, it would be helpful to have some of the additional use cases enumerated.

[zozo123]

Thanks @joshua-stauffer! Opened #11928 — it adds the use-case enumeration to the Ephemeral Data Context section and leaves the code snippet DataContext-focused as you suggested.