bson encoding of array fails

[nblumoe]

Results are not being stored to mongodb due to failing bson encoding:

...
Loaded data from building UK-DALE_building_5 for fold unseen_appliances from 2014-06-29 17:23:48+01:00 to 2014-09-07 15:02:18+01:00.
Done loading NILMTK mains data.
Directory exists = '/Users/nblu/Projects/16-01/src/neuralnilm/results/e577/ae/kettle'
  Re-using directory.
Traceback (most recent call last):
  File "./scripts/run_experiments.py", line 59, in <module>
    main()
  File "./scripts/run_experiments.py", line 21, in main
    _run_job(next_job)
  File "./scripts/run_experiments.py", line 47, in _run_job
    eval("{next_job}.run('{next_job}')".format(next_job=next_job))
  File "<string>", line 1, in <module>
  File "/Users/nblu/Projects/16-01/src/neuralnilm/experiment_definitions/e577.py", line 84, in run
    report = trainer.submit_report()
  File "build/bdist.macosx-10.12-x86_64/egg/neuralnilm/trainer.py", line 168, in submit_report
  File "/Users/nblu/.virtualenvs/neuralnilm/lib/python2.7/site-packages/pymongo/collection.py", line 657, in insert_one
    bypass_doc_val=bypass_document_validation),
  File "/Users/nblu/.virtualenvs/neuralnilm/lib/python2.7/site-packages/pymongo/collection.py", line 562, in _insert
    check_keys, manipulate, write_concern, op_id, bypass_doc_val)
  File "/Users/nblu/.virtualenvs/neuralnilm/lib/python2.7/site-packages/pymongo/collection.py", line 543, in _insert_one
    check_keys=check_keys)
  File "/Users/nblu/.virtualenvs/neuralnilm/lib/python2.7/site-packages/pymongo/pool.py", line 424, in command
    self._raise_connection_failure(error)
  File "/Users/nblu/.virtualenvs/neuralnilm/lib/python2.7/site-packages/pymongo/pool.py", line 552, in _raise_connection_failure
    raise error
bson.errors.InvalidDocument: Cannot encode object: array([[[ 3.30032039]],

       [[ 0.16541311]],

       [[ 0.38603193]],

    ...

       [[ 0.20147485]],

       [[ 1.73182499]]], dtype=float32)

The issue might be that numpy number types are not natively supported by pymongo/mongodb. I am just surprised that this ever worked if that is the real issue.

I am going to try converting it manually before storing to mongodb.

[JackKelly]

Yeah, to be brutally honest, this code base is kind of only half finished! You might be better off building your own Neural Nets. Perhaps using something like TensoFlow

[nblumoe]

Ah, I was already planning on doing that next. Thought it would have been useful to run this first though and now this seems to be the final missing step. Might just be "sunk cost fallacy" though. ;)