Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions features/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -70,26 +70,33 @@ SOURCES := $(patsubst %.feature,%.textproto,$(FEATURES))

.DELETE_ON_ERROR:

all : $(FEATURES)
HELP_TEXT += " all creates all features\n"
.PHONY: all
all : $(FEATURES) git.log

HELP_TEXT += " %.textproto copies the .textproto from the cel-spec directory\n"
HELP_TEXT += " git.log creates a git.log file with the last 5 lines of the git log\n"
git.log :
cd $(CEL_SPEC_PATH) && git log --format=oneline --abbrev-commit -n 5 --decorate=full > $(HERE)$@

HELP_TEXT += " %.textproto copies the .textproto from the cel-spec directory\n"
%.textproto : $(CEL_SIMPLE_TESTDATA)/%.textproto
cp $(CEL_SIMPLE_TESTDATA)/$@ $@

HELP_TEXT += " %.feature generates Gherkin feature from %.textproto source\n"
HELP_TEXT += " %.feature generates Gherkin feature from %.textproto source\n"
%.feature : %.textproto
uv run $(TOOLS)/gherkinize.py -v $^ -o $@

HELP_TEXT += " clean cleans generated feature files\n"
HELP_TEXT += " clean-features cleans generated feature files\n"
.PHONY: clean-features
clean-features:
rm $(FEATURES)

HELP_TEXT += " clean cleans generated feature files and textproto sources\n"
HELP_TEXT += " clean cleans generated feature files and textproto sources\n"
.PHONY: clean
clean:
rm $(FEATURES)
rm $(SOURCES)
-rm $(FEATURES)
-rm $(SOURCES)
-rm git.log

.PHONY: conformance-feature-list
conformance-feature-list:
Expand Down
38 changes: 28 additions & 10 deletions features/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ Acceptance Test Suite
We start with https://github.com/google/cel-spec/tree/master/tests/simple/testdata
as the acceptance test suite.

These files are captured as of commit 9f069b3e.
This is from May 5, 2025, version 0.24.0.
See the ``git.log`` file for the last 5 log entries in the ``google/cel-spec`` project.

We parse the text serialization of protobuf files to create Gherkin test scenarios.

Expand All @@ -31,17 +30,23 @@ As an alternative, this can be done by running **behave** manually, using the ``

::

PYTHONPATH=src uv run behave features
PYTHONPATH=src uv run behave -Denv="py312" features

::

PYTHONPATH=src uv run behave -Denv="py312" features/json_query.feature

The value of ``env`` should match the default Python in your virtual environment.
This is required to test the ``json_query.feature``.

To run a subset of the features, pick a specific file.

::

PYTHONPATH=src uv run behave features/basic.feature

To run the ``@wip`` tests, include ``--tags=wip``.
To skip the Work-in-Process features, include ``--tags=~wip``.

To skip the ``@wip`` tests, include ``tags=~wip``.

Building the Conformance Test Features
======================================
Expand All @@ -51,17 +56,30 @@ See the ``tools/README.rst`` for more information on running this application.

Here's the bigger picture workflow:

1. Create a ``google/cel-spec`` project parallel to this project's directory.
1. Create a ``../google/cel-spec`` project parallel to this project's directory.
This should be based on https://github.com/google/cel-spec

(Or, create it anywhere and set ``CEL_SPEC_PATH`` to refer to this directory.)

2. Find the tag for the most recent release (e.g. ``v0.24.0``)
2. Run the ``tools/refresh_spec.``py to find the most recent tag and pull the current files.

5. Run ``make`` to copy the ``.textproto`` files to this directory and create ``.feature`` files from them.
This will **also** create a ``git.log`` file with the last 5 log entries to help pinpoint
the commit on which the acceptance test suite is based.

3. Do ``git pull tag v0.24.0`` to get the released version.
Remove the ``.textproto`` and ``.feature`` files and rebuild them:

4. Update this ``README.rst`` to reflect the version tag actually used.
::

5. Run ``make`` to copy the ``.textproto`` files to this directory and create ``feature`` files from them.
make clean all

Changing the ``@wip`` tags in the feature files.

The ``@wip`` tags are added by ``gherkinize.py`` based on a ``wip.toml`` configuration file.
As features are added, update the ``wip.toml`` file and rebuild the ``.feature`` files, without touching the ``.textproto`` files.

::

make clean-features all

This will reset the tags in the ``.feature`` files.
26 changes: 26 additions & 0 deletions features/bindings_ext.feature
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,29 @@ Scenario: bind/macro_not_exists
When CEL expression 'cel.bind(valid_elems, [1, 2, 3], ![4, 5].exists(e, e in valid_elems))' is evaluated
Then value is celpy.celtypes.BoolType(source=True)

@wip
Scenario: bind/shadowing

Given type_env parameter "x" is celpy.celtypes.IntType
and bindings parameter "x" is celpy.celtypes.IntType(source=1)
When CEL expression 'cel.bind(x, 0, x == 0)' is evaluated
Then value is celpy.celtypes.BoolType(source=True)

@wip
Scenario: bind/shadowing_namespace_resolution

Given type_env parameter "com.example.x" is celpy.celtypes.IntType
and bindings parameter "com.example.x" is celpy.celtypes.IntType(source=1)
and container is 'com.example'
When CEL expression 'cel.bind(x, 0, x == 0)' is evaluated
Then value is celpy.celtypes.BoolType(source=True)

@wip
Scenario: bind/shadowing_namespace_resolution_selector

Given type_env parameter "com.example.x.y" is celpy.celtypes.IntType
and bindings parameter "com.example.x.y" is celpy.celtypes.IntType(source=1)
and container is 'com.example'
When CEL expression "cel.bind(x, {'y': 0}, x.y == 0)" is evaluated
Then value is celpy.celtypes.BoolType(source=True)

59 changes: 59 additions & 0 deletions features/bindings_ext.textproto
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,63 @@ section: {
bool_value: true
}
}
test: {
name: "shadowing"
expr: "cel.bind(x, 0, x == 0)"
type_env: {
name: "x"
ident: {
type: { primitive: INT64 }
}
}
bindings: {
key: "x"
value: {
value: { int64_value: 1 }
}
}
value: {
bool_value: true
}
}
test: {
name: "shadowing_namespace_resolution"
expr: "cel.bind(x, 0, x == 0)"
container: "com.example"
type_env: {
name: "com.example.x"
ident: {
type: { primitive: INT64 }
}
}
bindings: {
key: "com.example.x"
value: {
value: { int64_value: 1 }
}
}
value: {
bool_value: true
}
}
test: {
name: "shadowing_namespace_resolution_selector"
expr: "cel.bind(x, {'y': 0}, x.y == 0)"
container: "com.example"
type_env: {
name: "com.example.x.y"
ident: {
type: { primitive: INT64 }
}
}
bindings: {
key: "com.example.x.y"
value: {
value: { int64_value: 1 }
}
}
value: {
bool_value: true
}
}
}
2 changes: 0 additions & 2 deletions features/comparisons.feature
Original file line number Diff line number Diff line change
Expand Up @@ -350,13 +350,11 @@ Scenario: eq_literal/not_eq_dyn_int_null
When CEL expression 'dyn(1) == null' is evaluated
Then value is celpy.celtypes.BoolType(source=False)

@wip
Scenario: eq_literal/not_eq_dyn_list_null

When CEL expression 'dyn([]) == null' is evaluated
Then value is celpy.celtypes.BoolType(source=False)

@wip
Scenario: eq_literal/not_eq_dyn_map_null

When CEL expression 'dyn({}) == null' is evaluated
Expand Down
14 changes: 0 additions & 14 deletions features/conversions.feature
Original file line number Diff line number Diff line change
Expand Up @@ -170,25 +170,21 @@ Scenario: int/double_round_neg
When CEL expression 'int(-123.456)' is evaluated
Then value is celpy.celtypes.IntType(source=-123)

@wip
Scenario: int/double_truncate

When CEL expression 'int(1.9)' is evaluated
Then value is celpy.celtypes.IntType(source=1)

@wip
Scenario: int/double_truncate_neg

When CEL expression 'int(-7.9)' is evaluated
Then value is celpy.celtypes.IntType(source=-7)

@wip
Scenario: int/double_half_pos

When CEL expression 'int(11.5)' is evaluated
Then value is celpy.celtypes.IntType(source=11)

@wip
Scenario: int/double_half_neg

When CEL expression 'int(-3.5)' is evaluated
Expand Down Expand Up @@ -451,13 +447,11 @@ Scenario: uint/double
When CEL expression 'uint(3.14159265)' is evaluated
Then value is celpy.celtypes.UintType(source=3)

@wip
Scenario: uint/double_truncate

When CEL expression 'uint(1.9)' is evaluated
Then value is celpy.celtypes.UintType(source=1)

@wip
Scenario: uint/double_half

When CEL expression 'uint(25.5)' is evaluated
Expand Down Expand Up @@ -499,25 +493,21 @@ Scenario: bool/string_1
When CEL expression "bool('1')" is evaluated
Then value is celpy.celtypes.BoolType(source=True)

@wip
Scenario: bool/string_t

When CEL expression "bool('t')" is evaluated
Then value is celpy.celtypes.BoolType(source=True)

@wip
Scenario: bool/string_true_lowercase

When CEL expression "bool('true')" is evaluated
Then value is celpy.celtypes.BoolType(source=True)

@wip
Scenario: bool/string_true_uppercase

When CEL expression "bool('TRUE')" is evaluated
Then value is celpy.celtypes.BoolType(source=True)

@wip
Scenario: bool/string_true_pascalcase

When CEL expression "bool('True')" is evaluated
Expand All @@ -528,25 +518,21 @@ Scenario: bool/string_0
When CEL expression "bool('0')" is evaluated
Then value is celpy.celtypes.BoolType(source=False)

@wip
Scenario: bool/string_f

When CEL expression "bool('f')" is evaluated
Then value is celpy.celtypes.BoolType(source=False)

@wip
Scenario: bool/string_false_lowercase

When CEL expression "bool('false')" is evaluated
Then value is celpy.celtypes.BoolType(source=False)

@wip
Scenario: bool/string_false_uppercase

When CEL expression "bool('FALSE')" is evaluated
Then value is celpy.celtypes.BoolType(source=False)

@wip
Scenario: bool/string_false_pascalcase

When CEL expression "bool('False')" is evaluated
Expand Down
8 changes: 4 additions & 4 deletions features/dynamic.feature
Original file line number Diff line number Diff line change
Expand Up @@ -888,7 +888,7 @@ Scenario: struct/field_assign_proto2

Given container is 'cel.expr.conformance.proto2'
When CEL expression "TestAllTypes{single_struct: {'un': 1.0, 'deux': 2.0}}" is evaluated
Then value is TestAllTypes(single_struct=celpy.celtypes.MapType({'deux': celpy.celtypes.DoubleType(source=2.0), 'un': celpy.celtypes.DoubleType(source=1.0)}))
Then value is TestAllTypes(single_struct=celpy.celtypes.MapType({'un': celpy.celtypes.DoubleType(source=1.0), 'deux': celpy.celtypes.DoubleType(source=2.0)}))

Scenario: struct/field_assign_proto2_empty

Expand Down Expand Up @@ -927,7 +927,7 @@ Scenario: struct/field_assign_proto3

Given container is 'cel.expr.conformance.proto3'
When CEL expression "TestAllTypes{single_struct: {'un': 1.0, 'deux': 2.0}}" is evaluated
Then value is TestAllTypes(single_struct=celpy.celtypes.MapType({'deux': celpy.celtypes.DoubleType(source=2.0), 'un': celpy.celtypes.DoubleType(source=1.0)}))
Then value is TestAllTypes(single_struct=celpy.celtypes.MapType({'un': celpy.celtypes.DoubleType(source=1.0), 'deux': celpy.celtypes.DoubleType(source=2.0)}))

Scenario: struct/field_assign_proto3_empty

Expand Down Expand Up @@ -1290,7 +1290,7 @@ Scenario: value_struct/field_assign_proto2

Given container is 'cel.expr.conformance.proto2'
When CEL expression "TestAllTypes{single_value: {'un': 1.0, 'deux': 2.0}}" is evaluated
Then value is TestAllTypes(single_value=celpy.celtypes.MapType({'deux': celpy.celtypes.DoubleType(source=2.0), 'un': celpy.celtypes.DoubleType(source=1.0)}))
Then value is TestAllTypes(single_value=celpy.celtypes.MapType({'un': celpy.celtypes.DoubleType(source=1.0), 'deux': celpy.celtypes.DoubleType(source=2.0)}))

Scenario: value_struct/field_assign_proto2_empty

Expand All @@ -1314,7 +1314,7 @@ Scenario: value_struct/field_assign_proto3

Given container is 'cel.expr.conformance.proto3'
When CEL expression "TestAllTypes{single_value: {'un': 1.0, 'deux': 2.0}}" is evaluated
Then value is TestAllTypes(single_value=celpy.celtypes.MapType({'deux': celpy.celtypes.DoubleType(source=2.0), 'un': celpy.celtypes.DoubleType(source=1.0)}))
Then value is TestAllTypes(single_value=celpy.celtypes.MapType({'un': celpy.celtypes.DoubleType(source=1.0), 'deux': celpy.celtypes.DoubleType(source=2.0)}))

Scenario: value_struct/field_assign_proto3_empty

Expand Down
4 changes: 2 additions & 2 deletions features/fields.feature
Original file line number Diff line number Diff line change
Expand Up @@ -276,8 +276,8 @@ Scenario: qualified_identifier_resolution/qualified_identifier_resolution_unchec
Given disable_check parameter is True
and type_env parameter "a.b.c" is celpy.celtypes.StringType
and type_env parameter "a.b" is celpy.celtypes.MapType
and bindings parameter "a.b.c" is celpy.celtypes.StringType(source='yeah')
and bindings parameter "a.b" is celpy.celtypes.MapType({'c': celpy.celtypes.StringType(source='oops')})
and bindings parameter "a.b.c" is celpy.celtypes.StringType(source='yeah')
When CEL expression 'a.b.c' is evaluated
Then value is celpy.celtypes.StringType(source='yeah')

Expand All @@ -303,8 +303,8 @@ Scenario: qualified_identifier_resolution/ident_with_longest_prefix_check

Given type_env parameter "a.b.c" is celpy.celtypes.StringType
and type_env parameter "a.b" is celpy.celtypes.MapType
and bindings parameter "a.b.c" is celpy.celtypes.StringType(source='yeah')
and bindings parameter "a.b" is celpy.celtypes.MapType({'c': celpy.celtypes.StringType(source='oops')})
and bindings parameter "a.b.c" is celpy.celtypes.StringType(source='yeah')
When CEL expression 'a.b.c' is evaluated
Then value is celpy.celtypes.StringType(source='yeah')

Expand Down
5 changes: 5 additions & 0 deletions features/git.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
40e3bde (HEAD -> refs/heads/master, refs/remotes/origin/master, refs/remotes/origin/HEAD) Workaround inconsistent \? in proto textformat (#512)
cb51b41 (tag: refs/tags/v0.25.2) Add conformance test cases for null assignability around repeated fields and maps (#510)
8e848ac Networking extension conformance tests (#507)
8d28382 Add clarification on comprehension scoping rules. (#506)
121e265 Add automated BCR publishing on release tags (#498)
Loading
Loading