Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing Cloud files #9686

Merged
merged 33 commits into from
Apr 16, 2024
Merged

Writing Cloud files #9686

merged 33 commits into from
Apr 16, 2024

Conversation

radeusgd
Copy link
Member

Pull Request Description

Important Notes

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

  • The documentation has been updated, if necessary.
  • Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
  • All code follows the
    Scala,
    Java,
    and
    Rust
    style guides. In case you are using a language not listed above, follow the Rust style guide.
  • All code has been tested:
    • Unit tests have been written where possible.
    • If GUI codebase was changed, the GUI was tested when built using ./run ide build.

@radeusgd radeusgd self-assigned this Apr 12, 2024
@radeusgd radeusgd changed the base branch from develop to wip/radeusgd/9363-use-cloud-path-resolver April 12, 2024 10:53
Base automatically changed from wip/radeusgd/9363-use-cloud-path-resolver to develop April 12, 2024 13:03
@radeusgd radeusgd force-pushed the wip/radeusgd/9291-cloud-file-write branch from 36dbc7e to f65e1e5 Compare April 12, 2024 14:45
@radeusgd
Copy link
Member Author

radeusgd commented Apr 12, 2024

After adding some basic caching:

  • the Inter_Backend_File_Operations_Spec (without S3) got from 89s down to 49s,
    • and with S3 from ~5m to 3m 45s
  • the Enso_File_Spec went from 64s down to 30s.
    So the caching seems like it was quite beneficial.

Currently I set the cache timeout to 60 seconds which seems like a decent trade-off between keeping data up-to-date and improving performance. Most operations that access the same file multiple times finish in under 1 second anyway, a new request that takes 0.2-1.3s every 60s seems to not affect the performance awfully.

The cache TTL can be changed or caching can be disabled if needed.

@radeusgd radeusgd added CI: Ready to merge This PR is eligible for automatic merge CI: Clean build required CI runners will be cleaned before and after this PR is built. labels Apr 12, 2024
@radeusgd radeusgd removed the CI: Ready to merge This PR is eligible for automatic merge label Apr 15, 2024
@radeusgd radeusgd added the CI: Ready to merge This PR is eligible for automatic merge label Apr 15, 2024
@mergify mergify bot merged commit fda41cb into develop Apr 16, 2024
35 checks passed
@mergify mergify bot deleted the wip/radeusgd/9291-cloud-file-write branch April 16, 2024 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI: Clean build required CI runners will be cleaned before and after this PR is built. CI: Ready to merge This PR is eligible for automatic merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement basic support for Enso_File write
3 participants