Skip to content

Commit

Permalink
chore: Support enabling experimental features via env variables
Browse files Browse the repository at this point in the history
  • Loading branch information
Stranger6667 committed Oct 12, 2023
1 parent 66fcbcc commit d0036d0
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 6 deletions.
25 changes: 25 additions & 0 deletions docs/experimental.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ Experimental features offer a glimpse into upcoming functionalities and enhancem
Enabling Experimental Features
------------------------------

Schemathesis provides a few ways to enable experimental features: via the CLI, Python tests, and environment variables.

.. _experimental-cli:

In CLI
Expand Down Expand Up @@ -52,6 +54,18 @@ By doing this, all schemas loaded afterwards will automatically use the enabled

Enabling an experimental feature globally will affect all your tests. Use this feature with caution.

Using Environment Variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can also enable experimental features through environment variables. This is particularly useful for CI/CD pipelines or when you don't have direct control over CLI arguments or test code.

For example, to enable experimental support for OpenAPI 3.1:

.. code-block:: bash
export SCHEMATHESIS_EXPERIMENTAL_OPENAPI_3_1=true
This will enable the OpenAPI 3.1 experimental feature for any Schemathesis runs in the same environment.

Current Experimental Features
-----------------------------
Expand Down Expand Up @@ -87,6 +101,17 @@ In Python Tests
# Globally enable OpenAPI 3.1 experimental feature
schemathesis.experimental.openapi_3_1.enable()
.. _openapi-31-env-vars:

Using Environment Variables
~~~~~~~~~~~~~~~~~~~~~~~~~~~

To enable OpenAPI 3.1 support via environment variables, you can set:

.. code-block:: bash
export SCHEMATHESIS_EXPERIMENTAL_OPENAPI_3_1=true
For more details, join the `GitHub Discussion <https://github.com/schemathesis/schemathesis/discussions/1822>`_.

Stabilization of Experimental Features
Expand Down
6 changes: 5 additions & 1 deletion src/schemathesis/cli/callbacks.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,11 @@ def convert_stateful(ctx: click.core.Context, param: click.core.Parameter, value
def convert_experimental(
ctx: click.core.Context, param: click.core.Parameter, value: Tuple[str, ...]
) -> List[experimental.Experiment]:
return [feature for feature in experimental.GLOBAL_EXPERIMENTS.available if feature.name in value]
return [
feature
for feature in experimental.GLOBAL_EXPERIMENTS.available
if feature.name in value or os.getenv(feature.env_var, "").lower() in TRUE_VALUES
]


def convert_checks(ctx: click.core.Context, param: click.core.Parameter, value: Tuple[List[str]]) -> List[str]:
Expand Down
14 changes: 12 additions & 2 deletions src/schemathesis/experimental/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
class Experiment:
name: str
verbose_name: str
env_var: str
description: str
discussion_url: str
_storage: "ExperimentSet" = field(repr=False)
Expand All @@ -25,9 +26,16 @@ class ExperimentSet:
available: set = field(default_factory=set)
enabled: set = field(default_factory=set)

def create_experiment(self, name: str, verbose_name: str, description: str, discussion_url: str) -> Experiment:
def create_experiment(
self, name: str, verbose_name: str, env_var: str, description: str, discussion_url: str
) -> Experiment:
instance = Experiment(
name=name, verbose_name=verbose_name, description=description, discussion_url=discussion_url, _storage=self
name=name,
verbose_name=verbose_name,
env_var=f"{ENV_PREFIX}_{env_var}",
description=description,
discussion_url=discussion_url,
_storage=self,
)
self.available.add(instance)
return instance
Expand All @@ -45,11 +53,13 @@ def is_enabled(self, feature: Experiment) -> bool:
return feature in self.enabled


ENV_PREFIX = "SCHEMATHESIS_EXPERIMENTAL"
GLOBAL_EXPERIMENTS = ExperimentSet()

OPEN_API_3_1 = GLOBAL_EXPERIMENTS.create_experiment(
name="openapi-3.1",
verbose_name="OpenAPI 3.1",
env_var="OPENAPI_3_1",
description="Support for response validation",
discussion_url="https://github.com/schemathesis/schemathesis/discussions/1822",
)
13 changes: 10 additions & 3 deletions test/experimental/test_experiments.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

def test_experiments():
experiments = ExperimentSet()
example = experiments.create_experiment("Example", "", "", "")
example = experiments.create_experiment("Example", "", "FOO", "", "")

del experiments

Expand All @@ -17,10 +17,17 @@ def test_experiments():
assert not example.is_enabled


@pytest.mark.parametrize(
"args, kwargs",
(
((f"--experimental={OPEN_API_3_1.name}",), {}),
((), {"env": {OPEN_API_3_1.env_var: "true"}}),
),
)
@pytest.mark.openapi_version("3.0")
@pytest.mark.operations("success")
def test_enable_via_cli(cli, schema_url):
result = cli.run(schema_url, f"--experimental={OPEN_API_3_1.name}")
def test_enable_via_cli(cli, schema_url, args, kwargs):
result = cli.run(schema_url, *args, **kwargs)
assert result.exit_code == ExitCode.OK, result.stdout
assert "Experimental Features:" in result.stdout
assert OPEN_API_3_1.is_enabled

0 comments on commit d0036d0

Please sign in to comment.