Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Workspace files being written unnecessarily #3805

Closed
tfmorris opened this issue Apr 11, 2021 · 2 comments · Fixed by #4897 or #6026
Closed

Workspace files being written unnecessarily #3805

tfmorris opened this issue Apr 11, 2021 · 2 comments · Fixed by #4897 or #6026
Assignees
@tfmorris
Labels
persistence Issues about the way user data is saved on disk Type: Bug Issues related to software defects or unexpected behavior, which require resolution. workspace About data directory where user projects and metadata are stored
Milestone
3.8

Comments

@tfmorris
Copy link
Member

Current Results

It looks like all of the individual projects' metadata.json files are getting written every 5 minutes, even if the project hasn't been modified in any way - or even opened. This is both wasteful and error prone because it provides many more windows for project corruption when no file write is needed.

Expected Behavior

File updates should be minimized and restricted to those that are actually needed. Unmodified files should not be re-written.

Screenshots

Screen Shot 2021-04-10 at 8 20 00 PM

Versions

  • Operating System: macOS 11.2.3 Big Sure
  • Browser Version: Chrome Version 89.0.4389.90 (Official Build) (x86_64)
  • JRE or JDK Version: bundled
  • OpenRefine: 3.4.1
@tfmorris tfmorris added Type: Bug Issues related to software defects or unexpected behavior, which require resolution. Status: Pending Review Indicates that the issue or pull request is awaiting review by project maintainers or collaborators labels Apr 11, 2021
@ComgLq24
Copy link
Contributor

ComgLq24 commented May 27, 2022 •
edited

Hi @tfmorris @wetneb , I am interested in this issue, can you assign it to me to have a try?

@wetneb wetneb mentioned this issue May 30, 2022
Merged
@thadguidry thadguidry added workspace About data directory where user projects and metadata are stored and removed Status: Pending Review Indicates that the issue or pull request is awaiting review by project maintainers or collaborators labels Oct 6, 2022
wetneb added a commit to ComgLq24/OpenRefine that referenced this issue Apr 24, 2023
@wetneb @ComgLq24
Fix frequent writes to metadata files
b4b023e 
Closes OpenRefine#3805.

Co-authored-by: ComgLq24 <ComgLqWork@outlook.com>
wetneb added a commit that referenced this issue Apr 24, 2023
@ComgLq24 @wetneb
Fix problem with frequent writes to metadata files (#4897)
92cd213 
* Fix frequent writes to metadata files

Closes #3805.

Co-authored-by: ComgLq24 <ComgLqWork@outlook.com>

* Update after migration from LocalDateTime to Instant

---------

Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
@wetneb wetneb added this to the 3.8 milestone Apr 24, 2023
@tfmorris tfmorris reopened this Jul 3, 2023
@tfmorris tfmorris added the persistence Issues about the way user data is saved on disk label Jul 3, 2023
@tfmorris tfmorris assigned tfmorris and unassigned ComgLq24 Aug 25, 2023
tfmorris added a commit to tfmorris/OpenRefine that referenced this issue Aug 25, 2023
@tfmorris
Make sure lastSave timestamp is initialized on load - fixes OpenRefin…
6bf04dc 
…e#3805
@tfmorris tfmorris mentioned this issue Aug 25, 2023
Merged
@tfmorris
Copy link
Member Author

I reopened this because every project is getting written every time OpenRefine starts up. I've started a branch which fixes that and a few other issues.

tfmorris added a commit to tfmorris/OpenRefine that referenced this issue Sep 13, 2023
@tfmorris
Make sure lastSave timestamp is initialized on load - fixes OpenRefin…
4db0632 
…e#3805
tfmorris added a commit to tfmorris/OpenRefine that referenced this issue Sep 13, 2023
@tfmorris
Make sure lastSave timestamp is initialized on load - fixes OpenRefin…
5076d6d 
…e#3805
wetneb pushed a commit that referenced this issue Sep 25, 2023
@tfmorris
Don't write newly read projects before they're modified (#6026)
bbfdd9a 
* Restore previous constructor behavior

Use modified time if we're given one.

* Enhance test to cover initial read case

It wasn't testing if an unnecessary project
write occurs when projects are read, but
not written

* Make sure lastSave timestamp is initialized on load - fixes #3805

* Fix test for workspace save on project remove from PR #4796

* Save workspace when projects removed. Fixes #1418

* Refactor to clean up workspace save

- only write temp file if needed rather than writing,
  then deleting if unneeded
- check and log errors on file create/delete/rename
- use try-with-resources to avoid resource leaks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tfmorris tfmorris

Labels
persistence Issues about the way user data is saved on disk Type: Bug Issues related to software defects or unexpected behavior, which require resolution. workspace About data directory where user projects and metadata are stored
Projects
None yet
Milestone
3.8
4 participants
@tfmorris @wetneb @thadguidry @ComgLq24