Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrub dataclips during presentation #1606

Closed
4 of 5 tasks
NickOpenFn opened this issue Jan 10, 2024 · 0 comments · Fixed by #1617
Closed
4 of 5 tasks

Scrub dataclips during presentation #1606

NickOpenFn opened this issue Jan 10, 2024 · 0 comments · Fixed by #1617
Assignees

Comments

@NickOpenFn
Copy link

NickOpenFn commented Jan 10, 2024

Currently, we may expose secrets in the run results in the output dataclip. We will be implementing a rule that limits what the runtime writes to the run_result, but would like an additional failsafe of scrubbing this before displaying it to the user on the presentation layer:

Per the Slack conversation from @taylordowns2000: https://openfn.slack.com/archives/C0394D0GBUN/p1704798081537769?thread_ts=1703149940.085049&cid=C0394D0GBUN

just finished the call with Mtuchi and Joe around secrets getting stored in run_result dataclips. We think that (if the additional overhead isn’t too high) it makes sense to scrub dataclips during presentation (i.e., after the liveview gets the JSON from the db, after it converts it to a string, and finally right before it renders it in the code-viewers on run_viewer, attempts_live, and the dataclip viewer.
In addition to this, we’ll handle the actual object in the runtime, but it’s a very good failsafe to scrub the strings right before they’re sent to the HTML view. What do you think? (Roger, is this a terrible idea from a performance perspective? Could we hook up the existing scrubber (using the secrets list from an associated attempt) to scrub dataclips with type run_result once they’re converted to text and are about to be viewed as HTML?)

Notes:

  • Scrubbing should occur after the liveview receives the JSON from the database.
  • Scrubbing should take place after the JSON is converted to a string.
  • Integration with the existing scrubber should be implemented to utilize the secrets list from the associated attempt for scrubbing dataclips with type run_result.

Acceptance requirement:

@NickOpenFn NickOpenFn changed the title Scrubbing Scrub dataclips during presentation Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants