Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve CSV schema validation with filename patterns #22

Merged
merged 4 commits into from
Mar 13, 2024
Merged

Improve CSV schema validation with filename patterns #22

merged 4 commits into from
Mar 13, 2024

Commits on Mar 13, 2024

  1. Improve CSV schema validation with filename patterns 

    This commit introduces filename pattern validation to the CSV schema, which allows increased data consistency checks. It also extends the capability to include additional columns to the schema, providing a more flexible structure. Enhancements were also made to error handling, introducing a quick-stop feature to expedite error discovery.
    Denis Smet committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    778859c View commit details
    Browse the repository at this point in the history

  2. Fix filename pattern validation message for demo-3.csv

    Denis Smet committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    43f83d7 View commit details
    Browse the repository at this point in the history

  3. Update demo invalidating steps in workflow 

    Previously, the invalid CSV file steps in our workflows performed their tasks differently and continued on errors. Now, the mechanism has been streamlined, and errors will halt the execution. This change results in more accurate validation and error detection.
    Denis Smet committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    bcd9d11 View commit details
    Browse the repository at this point in the history

  4. Improve error handling in CsvFile 

    This update introduces the use of the Utils::cutPath function in the error reporting for filename pattern mismatch in CsvFile.php. The error message, now, only displays the relative file path instead of the absolute one. The changes help decrease verbosity and improve clarity in error messages raised during CSV file validation.
    Denis Smet committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    00bcb66 View commit details
    Browse the repository at this point in the history