Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create plots as part of github action & upload plots + files to S3 #10

Closed
2 tasks done
AlexanderViand opened this issue Jul 31, 2020 · 1 comment
Closed
2 tasks done
Assignees
Projects

Comments

@AlexanderViand
Copy link
Contributor

AlexanderViand commented Jul 31, 2020

  • Create a GitHub Action that creates vector + pixel versions of the plots and stores them in S3 bucket.
  • Upload plotting scripts from repository to S3 bucket after workflow run.

Originally posted by @pjattke in #1 (comment)

@pjattke pjattke added this to In progress in SoK Tasks Aug 4, 2020
pjattke pushed a commit that referenced this issue Aug 4, 2020
@pjattke
Copy link
Collaborator

pjattke commented Aug 5, 2020

Task completed!

  • The repository folder scripts/plotting contains all relevant files.
    • All plotting files (*.py) are copied from that repository folder during the workflow run to the plot/ folder in the S3 benchmark folder (e.g., 20200804_160933/plot).
  • The workflow run waits until all benchmark runs are completed and then starts the plotting generation.
    • Until now, the workflow did not know when the benchmark is finished. I added a loop until (VM state == running) { sleep 1 min } for blocking waiting.
    • As yesterday a workflow run aborted and the VM was running for 15 hrs, I added a timeout parameter that determines when the VM is automatically halted and also determines how long we will wait for the benchmark run to finish before we continue with the plot generation.
      • The halt command is sent to the VM as a background job that is invoked after the given timeout (i.e., 60 mins).
  • After all benchmark runs are completed, the plot generation job starts.
    • It generates the plots and saves them in S3 (all in Python code, see plot_all.py)

@pjattke pjattke closed this as completed Aug 5, 2020
@pjattke pjattke moved this from In progress to Done in SoK Tasks Aug 5, 2020
pjattke pushed a commit that referenced this issue Aug 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
SoK Tasks
  
Done
Development

No branches or pull requests

2 participants