Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor file access with a transactional file system #422

Merged
merged 9 commits into from Mar 5, 2019

Conversation

1 participant
@ubruhin
Copy link
Member

commented Feb 28, 2019

Description

This is a complete refactoring of the file system access of library elements and projects. See #172 for the original motivation and goal. Although this PR does not implement all tasks defined in #172, it's still a huge improvement compared to the current concept.

Maybe this fixes the issues reported in #390 and this forum thread, but I'm not completely sure because I couldn't reproduce these issues.

Implementation

A new class TransactionalFileSystem allows to perform file operations (list/read/write/add/remove) without writing them to the harddrive immediately. Instead, all operations are cached in memory and can be written all at once to the harddrive in an extremely safe way. The changes are (pseudo-)atomically written to disk to guarantee consistent project files in every case. See Atomic Saving Procedure for details.

In addition, it allows to create and restore periodic backups (autosave). See Automatic Periodic Saving for details.

Basically it does the same as we already have, but with several improvements:

  • Much more clean software architecture. Project related classes no longer need to know anything about backup/restore/autosave features, these are now all handled by the TransactionalFileSystem class. So the complexity of project related classes is reduced.
  • The safety should be higher now, i.e. LibrePCB should behave much more robust in case of application crashes or file access issues. For example the autosave backups should be very reliable now.
  • Much more future-proof because we can now perform file format upgrades in-memory, i.e. without writing an upgrade immediately to the harddrive. This is useful for users as they will be able to open projects with an older file format without the need of a one-way upgrade.

Overall, it was much harder than I initially thought, thus it cost me a lot of time to implement it 馃槥 That's the reason why there wasn't a lot of progress the last weeks... ;)

@ubruhin ubruhin added this to the 0.1.1 milestone Feb 28, 2019

@ubruhin ubruhin self-assigned this Feb 28, 2019

@ubruhin ubruhin changed the title Refactor file access with a transactional file system WIP: Refactor file access with a transactional file system Feb 28, 2019

@ubruhin ubruhin force-pushed the 172-transactional-file-system branch from 862f0d0 to 4f9bbb5 Mar 1, 2019

@ubruhin ubruhin added this to In Progress in Improve software architecture Mar 2, 2019

@ubruhin ubruhin force-pushed the 172-transactional-file-system branch 4 times, most recently from dc4a47c to ea10336 Mar 2, 2019

@ubruhin ubruhin changed the title WIP: Refactor file access with a transactional file system Refactor file access with a transactional file system Mar 3, 2019

@ubruhin ubruhin marked this pull request as ready for review Mar 3, 2019

@ubruhin ubruhin force-pushed the 172-transactional-file-system branch from ea10336 to cecc5de Mar 3, 2019

@ubruhin

This comment has been minimized.

Copy link
Member Author

commented Mar 3, 2019

I think this is ready now. I just hope there are no critical bugs left 馃槈

@ubruhin ubruhin force-pushed the 172-transactional-file-system branch from cecc5de to c455d60 Mar 3, 2019

@ubruhin ubruhin merged commit a8b0967 into master Mar 5, 2019

4 checks passed

continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@ubruhin ubruhin deleted the 172-transactional-file-system branch Mar 5, 2019

@ubruhin ubruhin referenced this pull request Mar 5, 2019

Closed

Create a file system abstraction layer #172

6 of 9 tasks complete

@ubruhin ubruhin moved this from In Progress to Done in Improve software architecture Mar 5, 2019

ubruhin added a commit that referenced this pull request Mar 17, 2019

Merge pull request #422 from LibrePCB/172-transactional-file-system
Refactor file access with a transactional file system
(cherry picked from commit a8b0967)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can鈥檛 perform that action at this time.