Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

match more input filenames #11

Merged
merged 1 commit into from Aug 14, 2023

Conversation

hulet
Copy link
Contributor

@hulet hulet commented Aug 4, 2023

Description of change

It is rare that we will need to match an input filename in the format table_name followed by any number of period (.) characters followed by .csv as described by the regex submitted in d6b7c18.

The change in d6b7c18 also means that the pattern regex no longer matches the default DDS filename format of <table_name>.{change|all}.<sequence_number>.<creation_date>.csv e.g. TIMESHEET.change.26990.2023-08-02_08.31.43_UTC_del_00001.csv.

This change properly updates the regex pattern to match any file name beginning with the table_name and ending in .csv which may or may not have additional content in between.

Manual QA steps

  • Upload sample files to the S3 bucket, e.g.
    • TIMESHEET.change.26990.2023-08-02_08.31.43_UTC_del_00001.csv
    • TIMESHEETENTRY.change.26683.2023-07-27_08.26.28_UTC_cr_00000.csv
    • Encor Revenues Inception To Date.csv
  • With a valid config.json run tap-intacct -c config.json --discover
  • Verify that all uploaded files are Found but that the TIMESHEETENTRY file is not processed together with the TIMESHEET file.

Risks

  • Not deploying this means that DDS exported files are no longer visible to the tap

Rollback steps

  • revert this branch

match input filenames with content between the table name and .csv
suffix
@rdeshmukh15
Copy link
Contributor

@hulet : thanks for your contribution. I will review the change and get it deployed.

@rdeshmukh15 rdeshmukh15 self-requested a review August 14, 2023 11:45
@rdeshmukh15 rdeshmukh15 merged commit c6f81ee into singer-io:master Aug 14, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants