Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid versioning issues with report data #141

Open
rbriski opened this issue Jun 4, 2020 · 0 comments
Open

Avoid versioning issues with report data #141

rbriski opened this issue Jun 4, 2020 · 0 comments

Comments

@rbriski
Copy link
Contributor

rbriski commented Jun 4, 2020

Currently, if the data structure of a report changes significantly, it could introduce bugs for existing reports. For example, if fields are renamed, removed or added to the report entity.

This is something that's bound to happen over the lifetime of this plugin. We could do one of two things:

  1. Provide upgrade scripts for each version change
  2. Allow data structures to work across upgrades

I believe that the second option is the easiest way forward, specifically by using protocol buffers. Airflow will let us store base64 encoded data in a variable, which means we can store protobufs as report entity values, just like we do now. The difference is that, instead of JSON, they'll be binary.

An obvious con is that binary is not human readable. I would say, however, that the variable value isn't being read anyway. The main benefit of the variable value is the ability to copy and paste (or import/export) across environments. For example, if we wanted to promote a report from dev to prod, you just copy the variable. This would still work with protobuf binary data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
rb status plugin
  
Awaiting triage
Development

No branches or pull requests

1 participant