Skip to content

Fields within a nest should support non-identifier-like names for eval and query #176

@gitosaurus

Description

@gitosaurus

Feature request

Follow up to #174 . A NestedFrame may contain both base columns and nested columns (columns containing a nest of fields) which have non-identifier-like names, for example having spaces or dots within them. In Pandas, these names can be referenced by enclosing them in backticks. However, while NestedFrame.eval and NestedFrame.query support this feature for the top-level column names, they don't support it for the fields within a nest. For example, this code does not work, but should:

nf = NestedFrame(data={"dog": [1, 2, 3], "good dog": [2, 4, 6]}, index=[0, 1, 2])
nested = pd.DataFrame(
    data={"n/a": [0, 2, 4, 1, 4, 3, 1, 4, 1], "n/b": [5, 4, 7, 5, 3, 1, 9, 3, 4]},
    index=[0, 0, 0, 1, 1, 1, 2, 2, 2],
)
nf = nf.add_nested(nested, "bad dog")
nf.query("`bad dog`.`n/a` > 2")

The backtick-substitution is handled by the clean_column_name function from Pandas. While the resolution of nested fields should not be handled by Pandas eval resolvers (because they are not top-level identifiers), the NestedFrame code for resolving fields within a nest ought to use clean_column_name in an appropriate way, so that the syntactic extension is familiar.

The method NestedFrame._is_known_hierarchical_column and the property NestedFrame.all_columns are good places to start.

Before submitting
Please check the following:

  • I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
  • I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
  • If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions