Switch network adapter type to paravirtualized for increased performance

[hackdna]

Increase guest OS network performance by ~100x for faster S3 transfers and provisioning

[scottx611x]

I'm confused:
Do we ever talk to s3 from a local development environment? Re:~100x for faster S3 transfers

[hackdna]

We do now.

[scottx611x]

@hackdna Oh, okay I wasn't aware.

So someone couldn't properly develop locally if their network connection went down?
Does one have to run terraform commands after provisioning a vagrant box?

[codecov]

Codecov Report

Merging #2748 into develop will decrease coverage by 1.7%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #2748      +/-   ##
===========================================
- Coverage    57.92%   56.21%   -1.71%     
===========================================
  Files          411      411              
  Lines        26570    25692     -878     
  Branches      1250     1250              
===========================================
- Hits         15390    14443     -947     
- Misses       11180    11249      +69

Impacted Files	Coverage Δ
...ager/management/commands/process_metadata_table.py	0% <0%> (-88.24%)	⬇️
refinery/file_store/tasks.py	30.5% <0%> (-8.5%)	⬇️
refinery/file_store/models.py	71.87% <0%> (-7.59%)	⬇️
refinery/tool_manager/serializers.py	97.91% <0%> (-2.09%)	⬇️
...nery/data_set_manager/single_file_column_parser.py	79.65% <0%> (-1.17%)	⬇️
refinery/data_set_manager/tasks.py	43.23% <0%> (-0.88%)	⬇️
refinery/tool_manager/models.py	96.64% <0%> (-0.36%)	⬇️
refinery/tool_manager/tests.py	99.82% <0%> (-0.07%)	⬇️
refinery/data_set_manager/tests.py	99.76% <0%> (-0.01%)	⬇️
refinery/data_set_manager/views.py	44.63% <0%> (+0.21%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d5d55e...fa0f71c. Read the comment docs.

[hackdna]

Well, it's not really possible to develop "locally" anymore because a lot of tests require an Internet connection.

Regardless, the file storage backend can be specified using DEFAULT_FILE_STORAGE Django setting. Default is SymlinkedFileSystemStorage (derived from django.core.files.storage.FileSystemStorage) in Vagrant and it does not require an Internet connection.

The only requirement to enable the new functionality in this PR is to run vagrant reload.

[scottx611x]

Okay sure, I think I was more confused about your comment here: #2748 (comment)... But you're just switching the storage backend to S3Storage to do some file_store specific testing, correct? It's not something we normally do or are required to do, but something we can do if necessary.

Do you think it would be useful to have some notes on how one could configure their Vagrant instance to be able to talk to s3? Its a little more involved than switching the storage backend, no?

Why not just bring up an AWS stack to test such functionality?

[hackdna]

The change in this PR is completely backward-compatible and does not require any additional configuration or updates. Once S3 storage backend support is fully functional it will also have all the necessary documentation. Developing on an EC2 instance is a pain for reasons that should be obvious.

[scottx611x]

Sure. Was just curious from the initial mention of talking to s3 from Vagrant and was trying to learn more.

Still not sure what benefit using the S3 storage backend from a Vagrant environment gives us over using an environment that is more similar to a production one.

[hackdna]

Happy to discuss this in depth at the next dev meeting.