v1.11.0

Lookup

The lookup wrangle is now widely available. This can be used to look up data from a saved lookup wrangle. Lookups support key (exact) or semantic (most similar meaning) based matches.

wrangles:
  - lookup:
      input: column_to_be_looked_up
      output:
        - column1_from_lookup
        - column2_from_lookup
      model_id: <id>

Batch

Added a batch wrangle. This can be used to execute a series of wrangles broken into a series of batches. The batches can optionally be executed in parallel with a threads parameter, and provide an error output to catch errors.

wrangles:
  - batch:
      batch_size: 10
      # threads: 5
      # on_error:
      #  error_column: error_value
      wrangles:
        - ...

Others

• Fixed matrix set to preserve the order of values from the input column(s).
• Use gpt-4o for GPT based tests.
• select.sample: Allow strings for the rows parameter if they are valid numbers.
• Fix schema input/output for fraction_to_decimal to show arrays are allowed.
• Fix a bug where accordion would introduce an empty string for inputs containing empty lists.
• Improved devcontainer definition for codespaces.
• Explode: added a drop_empty parameter. If true, empty lists will not produce a row in the exploded output.
• Pass any additional parameters for cloud based wrangles as url params.
• If an empty list is passed to any server based wrangles, such as in the case of a where filtering all rows, shortcut and return an empty list immediately.
• Improve handling of special characters in variable names. All non-alphanumeric characters are replaced by underscores or available within the kwargs dict using the original name.
• Enable where to work in conjunction with wrangles that use special output syntax like extract.ai and lookup.
• Fix a bug that where could cause data loss for falsy values.