Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mount NFS persistent volumes for local to database benchmark #945

Merged
merged 12 commits into from
Oct 19, 2022

Conversation

sunank200
Copy link
Contributor

Description

What is the current behavior?

We would like to run benchmarking script for loading local files. For that, the datasets need to be locally available.

closes: #883

What is the new behavior?

  • Create Persistent Disk in Google Compute Engine
  • Create NFS Server in GKE
  • Create NFS Service
  • Create Persistent Volume and Persistent Volume Claims
  • Configuring Pods with PVC

Does this introduce a breaking change?

No

Checklist

  • Created tests which fail without the change (if possible)
  • Extended the README / documentation, if necessary

@codecov
Copy link

codecov bot commented Oct 19, 2022

Codecov Report

Base: 93.48% // Head: 90.29% // Decreases project coverage by -3.19% ⚠️

Coverage data is based on head (67e8ee0) compared to base (1aa0900).
Patch has no changes to coverable lines.

❗ Current head 67e8ee0 differs from pull request most recent head efff09b. Consider uploading reports for the commit efff09b to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #945      +/-   ##
==========================================
- Coverage   93.48%   90.29%   -3.20%     
==========================================
  Files          64       12      -52     
  Lines        2947      443    -2504     
  Branches      324       42     -282     
==========================================
- Hits         2755      400    -2355     
+ Misses        131       31     -100     
+ Partials       61       12      -49     
Impacted Files Coverage Δ
python-sdk/src/astro/files/types/__init__.py
python-sdk/src/astro/files/locations/base.py
python-sdk/src/astro/files/__init__.py
python-sdk/src/astro/sql/operators/dataframe.py
python-sdk/src/astro/lineage/extractor.py
python-sdk/src/astro/sql/operators/export_file.py
python-sdk/src/astro/__init__.py
python-sdk/src/astro/files/locations/local.py
...thon-sdk/src/astro/sql/operators/base_decorator.py
python-sdk/src/astro/lineage/facets.py
... and 42 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@sunank200 sunank200 merged commit da772e4 into main Oct 19, 2022
@sunank200 sunank200 deleted the persistent-volumes branch October 19, 2022 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create and Use docker volume to save all the datasets
5 participants