New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider moving the data storage to eg. sqlite #3

Closed
benzea opened this Issue Nov 23, 2012 · 4 comments

Comments

Projects
None yet
1 participant
@benzea
Member

benzea commented Nov 23, 2012

Currently the data is stored by simply pickeling it. This works really great, but my guess is that this is a limitation in some use cases (you need to create a new project, etc. every time).

A first step to make SDAPS more flexible in this regard would be to move to an SQLite based storage per project. Once that works, one could consider using only one database for multiple projects.

Doing this could make it easier/possible to automatically sort scanned data into the correct project.

@benzea

This comment has been minimized.

Member

benzea commented Aug 27, 2014

Maybe some comments (as there was interest by some party to work on it).

  • There is a main object tree representing the questionnaire metadata. It needs to be serialized/deserialized to a database. (survey.questionnaire, objects in the questionnaire.py file)
  • Data is mostly accessed indirectly using the index from the global survey object. One solution might be to load/store data whenever the index changes.
  • Per image/questionnaire (called sheet internally) data storage objects are in sheet.py and data.py. These need to be loaded/stored when the index changes.
  • The only code that seems to access the sheets lists directly seems to be the reorder module. That will need to be changed at some point (but it is fine if it is broken for a while)
@benzea

This comment has been minimized.

Member

benzea commented Aug 27, 2014

Oh, and

  • have a look at how data is accessed (trough DataObject). You may of course modify this as you want.
@benzea

This comment has been minimized.

Member

benzea commented Dec 10, 2015

One more note, as I was just thinking about it. For a start we could only move the per-sheet data storage over into sqlite. The questionnaire/survey information could still stay in the current format.

That really means that we may only need a simple (sheet, id, key, value) storage system that supports values of different data types.

@benzea benzea added this to the 2.0 milestone Jan 16, 2018

@benzea

This comment has been minimized.

Member

benzea commented Apr 2, 2018

This has now happened. It just dumps json seralised objects into the database. One for the questionnaire and one for each "sheet".

@benzea benzea closed this Apr 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment