Optimize CSV validation and header mapping logic if header is enabled #115
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit addresses several changes in the CSV validation and header mapping logic. Firstly, a method to set IDs has been added in Column.php. Also, handling of headers in CSV files has been updated to better support files with and without headers. In addition, mapping of columns by header names has been moved to CsvFile.php from Schema.php for better alignment with data flow. Lastly, unnecessary exception throwing in getColumn method of Schema.php has been removed. Unit tests have also been updated to reflect these changes.
This commit includes implementing integer conversion for index mapping in CsvFile.php, which improves the header mapping logic. Furthermore, the header specification 'csv: header: false' has been removed from various test yaml files, implying a shift towards handling CSV files with headers. The changes collectively lead to a considerably streamlined CSV data processing workflow.
Pull Request Test Coverage Report for Build 8493418198Details
💛 - Coveralls |
This commit revamps the Makefile's BENCH_SCHEMAS variable to provide dynamic benchmarks selection. It also adjusts the benchmark-running targets to accommodate this change. These changes allow running different sets of benchmarks, improving overall benchmarking efficiency and customizability.
This change introduces a new attribute, 'name', in column rules within various Benchmark files. It improves clarity and identification of rule sets by assigning each with a distinct 'name'. This adjustment enhances the overall testing practice and results in better data organization and manipulation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit addresses several changes in the CSV validation and header mapping logic. Firstly, a method to set IDs has been added in Column.php. Also, handling of headers in CSV files has been updated to better support files with and without headers. In addition, mapping of columns by header names has been moved to CsvFile.php from Schema.php for better alignment with data flow. Lastly, unnecessary exception throwing in getColumn method of Schema.php has been removed. Unit tests have also been updated to reflect these changes.