Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Exponential explosion" when trying to upload 2 line CSV #2277

Open
mccalluc opened this issue Oct 25, 2017 · 1 comment
Open

"Exponential explosion" when trying to upload 2 line CSV #2277

mccalluc opened this issue Oct 25, 2017 · 1 comment

Comments

@mccalluc
Copy link
Member

  • Specific code commit: d38546b
  • Version of the web browser and OS: mac os
  • Environment where the error occurred (Vagrant VM and site conf mode or AWS instance): dev for chuck and scott

Steps to reproduce

Please list all the actions and the input data used:

  1. Try to import this CSV:
ID,Download_Url,Species
1,http://hgdownload.cse.ucsc.edu/goldenPath/hg19/encodeDCC/wgEncodeBroadHistone/wgEncodeBroadHistoneHuvecCtcfStdAlnRep3.bam,HomoSapiens

Observed behavior

  • In the UI: "Import of this data set failed. Please ensure that the structure of your metadata file matches that of the example metadata file."
  • In the console:
TypeError at /data_set_manager/import/metadata-table-form/
RuntimeError('Exponential explosion! Creation of 2 annotated nodes for 1 nodes of type Raw Data File is stopped!',) is not JSON serializable
  • In the log:
2017-10-25 14:45:02 INFO     data_set_manager.single_file_column_parser:183 run() - trying to add pre-isa archive file bam-test-one-liner.csv
2017-10-25 14:45:02 INFO     file_store.tasks:45 create() - Creating FileStoreItem using source 'bam-test-one-liner.csv'
2017-10-25 14:45:02 INFO     file_store.models:375 set_filetype() - File type is set to 'csv'
2017-10-25 14:45:02 INFO     file_store.tasks:54 create() - FileStoreItem created with UUID 1deecb7b-9b7b-4c31-b606-4bf040cf4aae
2017-10-25 14:45:02 DEBUG    file_store.tasks:69 import_file() - Importing FileStoreItem with UUID '1deecb7b-9b7b-4c31-b606-4bf040cf4aae'
2017-10-25 14:45:02 ERROR    file_store.tasks:145 import_file() - Could not open URL 'bam-test-one-liner.csv': 'Invalid URL 'bam-test-one-liner.csv': No schema supplied. Perhaps you meant 
http://bam-test-one-liner.csv?'
2017-10-25 14:45:02 DEBUG    data_set_manager.single_file_column_parser:194 run() - Parsing with file column 1 and auxiliary file column None
2017-10-25 14:45:02 INFO     file_store.tasks:45 create() - Creating FileStoreItem using source 'http://hgdownload.cse.ucsc.edu/goldenPath/hg19/encodeDCC/wgEncodeBroadHistone/wgEncodeBroad
HistoneHuvecCtcfStdAlnRep3.bam'
2017-10-25 14:45:02 INFO     file_store.models:375 set_filetype() - File type is set to 'bam'
2017-10-25 14:45:02 INFO     file_store.tasks:54 create() - FileStoreItem created with UUID 49489df6-9eb8-4ab9-97ba-1c34d4ff06bf
2017-10-25 14:45:02 DEBUG    file_server:449 _add_bam() - TDF file UUID was not provided
2017-10-25 14:45:02 INFO     file_server:459 _add_bam() - BAMItem created
2017-10-25 14:45:02 ERROR    data_set_manager.utils:470 update_annotated_nodes() - Exponential explosion! Creation of 2 annotated nodes for 1 nodes of type Raw Data File is stopped!
2017-10-25 14:45:02 ERROR    django.request:231 handle_uncaught_exception() - Internal Server Error: /data_set_manager/import/metadata-table-form/
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/refinery-platform/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/vagrant/.virtualenvs/refinery-platform/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/refinery-platform/local/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/refinery-platform/local/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/vagrant/refinery/data_set_manager/views.py", line 554, in post
    json.dumps({'error': error_msg}), 'application/json'
  File "/usr/lib/python2.7/json/__init__.py", line 243, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: RuntimeError('Exponential explosion! Creation of 2 annotated nodes for 1 nodes of type Raw Data File is stopped!',) is not JSON serializable
Internal Server Error: /data_set_manager/import/metadata-table-form/
@mccalluc mccalluc added the bug label Oct 25, 2017
@scottx611x
Copy link
Member

The RuntimeError should also be caught in data_set_manager/views.py and it's message should be returned by the view with a 400 response.

mccalluc added a commit that referenced this issue Oct 31, 2017
mccalluc added a commit that referenced this issue Nov 6, 2017
* Stub test

* Test the current failing behavior. TODO: Get the test code itself fixed, and then address #2277

* And another test for #2296

* Failed on travis, because of underlying dependency on rabbit.

* Remove process_default_table: Fix #2296

* Fix the fixtures

* division to multiplication, and check > 1

* More usual name. Trivial test: needs assertions

* test_update_annotated_nodes: Not the behavior I expected [skip ci]

* Meaningful assertions

* Ugly kludge to get passing tests

* ... and on travis rabbit is not running
@hackdna hackdna added this to the Next milestone Mar 15, 2018
@jkmarx jkmarx removed this from the Next milestone Apr 9, 2018
@hackdna hackdna added this to the Next milestone Oct 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants