Skip to content

Use separate mounted disk for persistence #152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
21 tasks done
Tracked by #71
mihaip opened this issue Apr 8, 2023 · 0 comments
Closed
21 tasks done
Tracked by #71

Use separate mounted disk for persistence #152

mihaip opened this issue Apr 8, 2023 · 0 comments
Labels
Fidelity Emulation fidelity (vs native builds and/or real hardware)

Comments

@mihaip
Copy link
Owner

mihaip commented Apr 8, 2023 •
edited
Loading

Instead of relying on ExtFS (which is not available on Mini vMac/System 6 and older), there could be a separate disk that gets mounted that has its contents persisted. Would rely on the changes made for #56 to have more direct hooks when reads/write happen.

  • Use Origin private file system to save changes to chunked files
  • Special-case chunks that are all 0 (by content hash and/or changing the hash to 0) and avoid having to load them at all
  • Handle emulator being open in multiple tabs (don't mount Saved HD there)
  • Request that storage is persistent
  • Add reset
  • Add import
  • Add export
  • Add export to .dsk
  • Add custom icon for Saved HD (life preserver?)
  • Tweak icon to be inset by 1 pixel on each edge
  • Add Readme that explains what Saved HD is
  • Allow custom instance with just Saved HD and no other disks, so that it can be booted from
  • Add OS installer disk images to CD-ROM library, so that it's easier to make a bootable disk
  • Check behavior in Safari (including private browsing windows)
  • Check behavior in Firefox
  • Check behavior in older browsers (which may not have createSyncAccessHandle and other APIs)
  • Enable by default (TODOs in run-def.ts and Custom.tsx)
  • Updated data saving references in in Stickies
  • Add to changelog
  • Add warning dialog if there are files in the "Saved" folder in The Outside World
  • A month later, remove support for old "Saved" folder in The Outside World
    Batch chunk writes (via periodicTasks?) and also call flush() at that time -> not worth it, risk of data loss
@mihaip mihaip added the Fidelity Emulation fidelity (vs native builds and/or real hardware) label Apr 8, 2023
@mihaip mihaip mentioned this issue Apr 8, 2023
Closed
32 tasks
mihaip added a commit that referenced this issue Apr 15, 2023
@mihaip
Switch to more direct disk API for the emulators
9e361b9 
Instead generating fiels into the virtual Emscripten file system
(that the emulators then read using POSIX APIs), expose a
workerApi.disks.* API that the emulators can use (implemented in
mihaip/minivmac@e351655 and
mihaip/macemu@4824c50).

This allows lazy loading to be handled more directly, without a
pre-allocated array of the size of the disk image. There is
now a EmulatorWorkerDisk interface with two separate implementations
(EmulatorWorkerChunkedDisk, EmulatorWorkerUploadDisk), and more
can be plugged in (e.g. for #152).

Fixes #56
@mihaip mihaip mentioned this issue May 12, 2023
Closed
mihaip added a commit that referenced this issue Aug 18, 2023
@mihaip
Add support for a persisted disk image
717e26c 
Add a mode where we persist the changes made to a chunked disk image in
the origin private file system feature that is supported by modern browsers.
They also support being able to get a synchronous file handle, which allows
us to directly intercept read/write calls from the emulator with no complications.

The persistence is handled as an overlay on top of the chunked data, we have a
separate dirty chunks bitmask that stores which chunks have been modified. When
we're about to read from a chunk, we first load it from the persisted disk.
Similarly, when a change to a chunk is made, we mark it as dirty and write it
to the persisted disk. This does mean that there is some write amplification
(any change writes a whole 256K chunk), but in practice performance appears to
be OK (the native file system cache kicks in presumably).

Since the disk image is mostly empty, we stored it compressed in Git (to avoid
having a large file). We also optimize empty chunks, so that we don't need to
store a signature for them, or load them over the network.

To feature detect if the browser supports the origin private file system and
synchronous file handles, we need to create a temporary worker (FileSystemSyncAccessHandle
is only ever defined in workers). We cache the result of this check in
localStorage so that we don't need to do it too often.

Currently disabled by default, but accessible via the custom instance
dialog or the ?saved_hd=true query parameter

Updates #152
mihaip added a commit that referenced this issue Aug 20, 2023
@mihaip
Move settings dialog to its own file
c66dd5f 
It'll be getting more complex if we're adding settings related to persistence.

Updates #152
mihaip added a commit that referenced this issue Aug 20, 2023
@mihaip
Add UI for requesting storage persistence
f3a1125 
I looked into doing this automatically (e.g. when there's a click in
the emulator), but persistence does not seem to be granted easily in
Chrome, so I'm letting it be explicit for now.

Updates #152
mihaip added a commit that referenced this issue Aug 20, 2023
@mihaip
Inset Saved HD icon by 1 pixel on each edge
33f9051 
The old version of the lifesaver icon was edge-to-edge, but that ends
up touching the name/label in the Finder, which does not look good.

Updates #152
mihaip added a commit that referenced this issue Aug 20, 2023
@mihaip
Add the ability to reset the saved disk storage
87ee8be 
To simplify the implementation this is done in the the UI process. We
need to stop the emulator (so that it doesn't try to write to the disk
and picks up changes), and if we need to stop it, we might as well
do the reset in the UI process (where we can more easily use async/await)

Updates #152
mihaip added a commit that referenced this issue Aug 20, 2023
@mihaip
Add import/export for saved disk storage
4fc0ae4 
Done via a .zip that contains the dirty chunks and data file.

Updates #152
mihaip added a commit that referenced this issue Aug 21, 2023
@mihaip
Allow saved disks to be exported as .dsk disk images
ce65461 
The base image and dirty chunks are merged into a .dsk image that
can be loaded into other (native) emulators.

Updates #152
mihaip added a commit that referenced this issue Aug 24, 2023
@mihaip
Add some system software CD-ROMs
97cf52b 
Allows clean installs to be done onto Saved HD volumes.

Updates #152
Updates #166
mihaip added a commit that referenced this issue Aug 24, 2023
@mihaip
Allow custom instances to be booted with just the Saved HD
51cef25 
It's possible that it has a a system folder on it, and thus all that's
needed.

Updates #152
mihaip added a commit that referenced this issue Aug 24, 2023
@mihaip
Add a couple more legacy Mac OS install CDs
e2601e3 
Updates #152
mihaip added a commit that referenced this issue Aug 26, 2023
@mihaip
Flesh out Read Me from Saved HD
ac7daa7 
Also moves such intro texts out of import-disks.py and into a "strings"
subdirectory so that they're easier to edit.

Updates #152
mihaip added a commit that referenced this issue Aug 26, 2023
@mihaip
Update most dependencies to current versions
88b0dba 
Mostly out of hygiene, but also picks up TypeScript type definitionss
for createWritable on FileSystemFileHandle.

Updates #152
mihaip added a commit that referenced this issue Aug 26, 2023
@mihaip
Fix copy/paste bug that was causing Saved HD to be always enabled in …
432a5d1 
…URLs

Updates #152
mihaip added a commit that referenced this issue Aug 26, 2023
@mihaip
Fixed Saved HD being enabled by default prematurely
d952643 
Updates #152
mihaip added a commit that referenced this issue Aug 26, 2023
@mihaip
Separate out lack of API support from other saved disk errors.
e5995fc 
Updates #152
mihaip added a commit that referenced this issue Sep 10, 2023
@mihaip
Enable Saved HD by default
e7a5309 
Also reference it in the in-app documentation.

Updates #152
mihaip added a commit that referenced this issue Sep 10, 2023
@mihaip
Add deprecation warning for ”Saved” folder in The Outside World
52204d3 
Only shown if we restore any files to it.

Also fixes usePersistentState to actually persist changes -- it read
from localStorage but wrote to sessionStorage, so it never actually
persisted anything.

Updates #152
mihaip added a commit that referenced this issue Sep 16, 2023
@mihaip
Add Saved HD blog post image
c84d460 
Updates #152
@mihaip mihaip closed this as completed in 9b01d11 Oct 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Fidelity Emulation fidelity (vs native builds and/or real hardware)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mihaip