[Bug] Upload Error

[kaitj]

Upload of fcsv throws error with message below:

Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Unit test isn't catching error

@tkkuehn - please add log

[tkkuehn]

Logs:

2020-04-01T20:25:07.771524+00:00 app[web.1]: [2020-04-01 20:25:07,770] ERROR in app: Exception on /validator.html [POST] 2020-04-01T20:25:07.771545+00:00 app[web.1]: Traceback (most recent call last): 2020-04-01T20:25:07.771546+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app 2020-04-01T20:25:07.771547+00:00 app[web.1]: response = self.full_dispatch_request() 2020-04-01T20:25:07.771547+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request 2020-04-01T20:25:07.771548+00:00 app[web.1]: rv = self.handle_user_exception(e) 2020-04-01T20:25:07.771548+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception 2020-04-01T20:25:07.771549+00:00 app[web.1]: reraise(exc_type, exc_value, tb) 2020-04-01T20:25:07.771549+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise 2020-04-01T20:25:07.771550+00:00 app[web.1]: raise value 2020-04-01T20:25:07.771551+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request 2020-04-01T20:25:07.771551+00:00 app[web.1]: rv = self.dispatch_request() 2020-04-01T20:25:07.771551+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request 2020-04-01T20:25:07.771552+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args) 2020-04-01T20:25:07.771553+00:00 app[web.1]: File "/app/controller.py", line 253, in validator 2020-04-01T20:25:07.771553+00:00 app[web.1]: user_data = csv_to_json(io.StringIO(upload.stream.read().decode('utf-8'))) 2020-04-01T20:25:07.771554+00:00 app[web.1]: File "/app/model_auto.py", line 101, in csv_to_json 2020-04-01T20:25:07.771554+00:00 app[web.1]: parsed_version = re.findall("\d+\.\d+", version_line)[0] 2020-04-01T20:25:07.771562+00:00 app[web.1]: IndexError: list index out of range 2020-04-01T20:25:07.773206+00:00 app[web.1]: 10.69.117.24 - - [01/Apr/2020:20:25:07 +0000] "POST /validator.html HTTP/1.1" 500 290 "http://fidvalidator.herokuapp.com/validator.html" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.162 Safari/537.36"

[kaitj]

Just to add onto this - just tested it out locally. Upload works fine when testing locally, which might explain why the unit tests aren't catching any errors. Believe they're performed within the github actions (container?) environment.

[kaitj]

Turns out no error in the code! Upload test was done using a "fcsv" file that contained html instead of fiducials.

Results in 500 error.

@tkkuehn @Park-Patrick

[kaitj]

Error was encountered again at Brainweb. Somethign we discussed yesterday that maybe what we should be doing is chceking the first few lines of the file, and if it does not match what is expected, throw a meaningful error.

Additionally, validator (should not currently) be saving any of the contents of these files if error is encountered as it doesn't get to that stage.

[kaitj]

See #79, which addresses the issue of invalid content

[kaitj]

PR for checking the invalid content has been merged. There are two checks implemented here:

1. Whether it has a proper header - will throw an error if it does not
2. Whether it has proper contents in case it passes the header check - will throw an error along the lines of invalid row values!

Thanks @tkkuehn