What's Changed
This Release includes several large updates:
Reporting
updated template and readme for rule run status by @gerrycampion in #1635
update for execution status spaces by @gerrycampion in #1643
New Operations
1329: Add DatasetIsCustom operation to determine if a dataset is custom by @alexfurmenkov in #1580
Add minus operation for set difference to support CORE-000334 by @RakeshBobba03 in #1613
New Metadata & Rule Types
1316: New rule - value against library metadata by @alexfurmenkov in #1680
1047 define variable has method by @DmitryMK in #1664
Add define_dataset_variable_order metadata (#791) by @RakeshBobba03 in #1508
library_variable_has_codelist new metadata by @SFJohnson24 in #1628
1198: Add variable_max_size to variables metadata for enhanced data insights by @alexfurmenkov in #1679
CSV Data Readers
1558 csv data reader by @alexfurmenkov in #1640
datasets.csv open rules support by @SFJohnson24 in #1688
Operator Improvements
value_does_not_have_multiple_references operator rework by @SFJohnson24 in #1638
fix equal string parts operator value_is_literal functionality by @SFJohnson24 in #1620
#1654 added regex to is_inconsistent_across_dataset operation by @alexfurmenkov in #1658
#1652 wildcard replacement with domain on grouping_variables by @alexfurmenkov in #1674
make library_column_order a shared parent class as a move toward consistency by @gerrycampion in #1671
Bug Fixes
#1121 fix empty alias creating raising IndexError by @alexfurmenkov in #1646
#1442 fix dataset filtering when -dp is provided by @alexfurmenkov in #1645
1305: Fix type error in row number assignment to handle NaN values by @alexfurmenkov in #1673
1675: fix colliding file path by @alexfurmenkov in #1695
Fix jsonata execution status by @gerrycampion in #1707
fix codelist operations type inference for empty dataframes by @gerrycampion in #1709
Fix get_dataset by @gerrycampion in #1691
1669: stop execution when multiple --data options are provided by @alexfurmenkov in #1684
#1267 changed full path to original path in stdm reporting by @alexfurmenkov in #1618
whitespace column header removal by @SFJohnson24 in #1663
#1660 added numeric fields normalization for supp datasets by @alexfurmenkov in #1676
1699: get_parent_model_column_order calculation made only for unique rdomains and merged by @alexfurmenkov in #1711
984: define subversion workaround by @alexfurmenkov in #1697
1681: added @cached decorator to get_dataset by @alexfurmenkov in #1687
fixes by @SFJohnson24 in #1619
error handling and removed if conditional by @SFJohnson24 in #1670
Error Handling
#1616: User-friendly errors for missing/wrong Datasets tab and columns, standard validation, and library metadata lookup failures by @RakeshBobba03 in #1626
1182: Add custom exceptions for unsupported XPT format and enhance error handling in XPTReader by @alexfurmenkov in #1644
983 handle missing external dictionaries by @RakeshBobba03 in #1611
Resolve rule execution errors by @gerrycampion in #1653
CI/CD & Infrastructure
1604 automated ci workflows should run on push to main by @gerrycampion in #1642
Add GitHub Action to merge schema files with markdown descriptions by @Copilot in #1603
merge schema only runs manually. add a check schema to run in general by @gerrycampion in #1648
add pull-request perm to check schema markdown by @gerrycampion in #1668
schemas updated by @gerrycampion in #1662
Dispatch by @SFJohnson24 in #1693
dispatch action by @SFJohnson24 in #1704
make test suite more secure by @gerrycampion in #1698
revert test suite and prohibit fork runs by @gerrycampion in #1706
remove define_version by @gerrycampion in #1683
#1248 Specify define xml path in operations by @alexfurmenkov in #1649
Documentation
Update readme check_parameter.md by @SFJohnson24 in #1637
inconcistent_across_dataset readme by @SFJohnson24 in #165
Update .pre-commit-config.yaml by @SFJohnson24 in #1647
688: update tests by @alexfurmenkov in #1666
Full Changelog: v0.15...v0.16.0
