New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add all_vars view. #288

Merged
merged 2 commits into from Jun 21, 2017

Conversation

Projects
None yet
2 participants
@sp4ghet
Collaborator

sp4ghet commented Jun 20, 2017

Overview

This change adds a new view to the database environmental_data_point called all_vars.
It also changes the behaviour of the csv exporter (csv list func) to stop exporting desired variable datapoints. I consider this a bug fix but maybe someone thought that was a feature when they implemented it.

Motivation/Details

For the fermentabot we want to have CSV exports to get the fermentation data out from the bot.
The openag_ui has a feature for this, enabled through the couchdb list functions. However, the current implementation is only able to export CSV files for individual environmental_variables from a subset of all environmental_variables declared in openag_ui/openag-config.json. The list function will run over start_key~end_key documents in the by_variable view. This isn't ideal if you want multiple variables.
We want a way to export multiple variables, so I've added a new view which will filter out only variables that we want. These are hardcoded into the view map function. This view wouldn't be necessary if all environmental variables were useful pieces of information but that's not the case so we have to make this view.

Alternatives

  • Remove environmental variable types that aren't "real" environmental variables (eg. air_flush_on)
    • This is ideal, but this requires some coordination across multiple devs. In the interim I think this change is the best option.
  • Filter out data client-side and construct csv within openag_ui
    • This doesn't utilize the parallel processing optimization of map/reduce in couchdb, but it would mean we can continue referencing the config within openag_ui without adding another location for data structure.
  • Add config file in database, have openag_brain and openag_ui look at those configs
    • This seems out of scope for this change.
  • "all_vars" csv includes "garbage" data as well
    • This harms the UX of getting data out and ignores the benefits of map/reduce.
  • rosbag csv exporter added to API, calling that api.
    • This would create a rosbag file which would be significantly large and needs major changes to the REST API. It also goes against the architecture we have currently, which makes this choice not ideal.

Drawbacks

This hardcodes the variable names of what we think is "relevant" in the view function, which is kind of nasty. It requires the least changes and shouldn't break anything else though, so it is the most elegant option given time constraints for me right now.

@rbaynes

Looks good. Thank you for the nice PR description.

@rbaynes rbaynes merged commit 4df5a35 into develop Jun 21, 2017

@rbaynes rbaynes deleted the db_view branch Jun 21, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment