Added facility to transform into JSON #165
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
& associated support for steps they require
…ithout making some client calls
…to be "client" side
…inputs to be tested while preserving the prior logic
…reaming class, to create a 'client' class that could invoke the Streaming Validator being constructed. Specs have also been copied from validator spec verbatim Major refactoring has been done within initialise method, with reordering of certain method calls, creation of an instance variable to store 2 other variables which are related to the CSV objects state and a refactoring of the parse CSV method to fit in line with the implementation of a validator stream that is intended to consume string input. The remaining LOC in the parse_csv method attempt to match the Streaming Validator as closely as possible Currently errors are thrown by StringIO objects which four specs opt to use instead of String or File input
…validator specs shows that they both fail with the same error - it is a consequence of how StringIO works alongside CSV library
…w from the original validator spec that tested functions in a unit testing fashion
…ciently separated - all three of the other 'context headers' functions can pass without invoking the validate method,
…amer class to effect that specs using validator.valid? as an assertion will pass erroneously henceforth
…refactoring the validator class from a Client/Observer POV
…t capturing the row data of imported CSVs - this is the next significant refactor
…pec 53 - undesired behaviour
… to the normal behaviour - the ensure is never invoked - perhaps this is because the rescue occurs within an enumerable
…ating all the methods in validate and thus warping where the exception was raised
…hing placed correctly however the loop still terminates incorrectly
…branch and return to an earlier less convoluted version now that I understand how better to specify the problem
…to feature-streaming-validation # Conflicts: # lib/csvlint/streaming_validate.rb
… as a method it encapsulates the problems with doing enumerations of CSV content when rescuing the exceptions in such a way that they persist in the Error Builder object
I have rewritten the specs so that the modularity of methods within the streaming_validate class can be explored in cleaner way - adopting a convention of stream for a variable that is neither StringIO or File or CSV instance and data for a variable that matches any of the aforementioned classes
…tant as precedents for unit testing the classes methods without having to invoke an entire validation and because they expect arrays as arguments - this has consequences for what type of object can be passed to the streaming validate class
…rage, however this is earmarked as something to change
…row & column parsing: availing of instance rather than new in the constructor http://docs.ruby-lang.org/en/2.1.0/CSV.html#method-c-instance
…r than string However now multiple tests which should return errors are returning valid - I believe this is because of not catching CSVmalformed Exceptions
added 14 commits
October 6, 2015 21:31
Conflicts: lib/csvlint/validate.rb
all transformation tests passing
added 11 commits
October 25, 2015 10:01
so this is just about making csvlint support transformations layered over the top, not having them part of the core gem
|
Can you rebase this PR against master please @JeniT? There's a lot of commits in this that I squashed before merging into master. Or have https://github.com/theodi/csv2json and https://github.com/theodi/csv2rdf superseded this? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ideally this should be in a separate gem, but this branch does include changes to the core csvlint.rb code that enable transformation to be carried out without validation. So hoping we can merge for now and then take out the pieces that need to be taken out later.