Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yfs and FileSystem needs to be changed to support rename file and directory. #56

Open
kifhan opened this issue Sep 18, 2017 · 4 comments

Comments

@kifhan
Copy link
Collaborator

kifhan commented Sep 18, 2017

Description

MultiHack use this way to maintain files.

self.yfs.set(filePath, Y.Text)

but I found out this way you get really messy when you try to rename file or directory.

Because of this problem: Cannot change property's key name on y-map. You must delete obj from Y-map and insert again. But when you do that (delete and insert) you cannot use exist Y data type object (y-text, y-array, etc) because Y-Map set takes only TypeDefinition and creates new data type object.

Solution

My Solution is use two Y-map data.

filesystem_index: 'Map',
filesystem_nodes: 'Map'

Both filesystem_index and filesystem_nodes use randomly generated contentID as a key.
One contentID can access both file meta information (filesystem_index) and file content (filesystem_nodes).

Meta data looks like this.

self.yFSIndex.set(contentID, {
  name: filename,
  type: filetype,
  contentID: contentID,
  replydbID: replydbID,
  parentPath: parentPath
})

Feel free to check my implementation on Rellat.
I also fix few things on filesystem.js and main file (index.js).
One tricky thing is when you use _buildPath you can't guarantee to get right meta data at right timing.
So I made this sideway.

self.execWhenTargetSet = function (target, f) {
    function ft () {
      if (target) f()
      else setTimeout(ft, 50)
    }
    ft()
  }

I'm not sure this is best way but this works pretty well.

@t-mullen
Copy link
Member

I've talked to the YJS author about initializing values before, and it doesn't seem to be a planned feature. I don't think the algorithm can support it.

The overhead of pushing a project into YJS is huge because every character is an insert. :/

Your solution will work great for renames, but I wish we didn't need to do thousands of inserts every time a file is created.

@kifhan
Copy link
Collaborator Author

kifhan commented Sep 19, 2017

I see the point.
Like you said, it's a limit of OT and CRDT structure.

ex) Only google docs, sheet, drawing, etc support OT in google drive. If you have a docx file and want to collaborate with other, you must convert it to google docs type.

So I think inserting file string into YJS is a sort of converting. It's not the best but better then dropping CRDT.

I'm also thinking about using WebTorrent magnet address for large file (ex. over 100MB or Zip file) instead of Y data type. Like this one. P2P shared-filesystem (yjs)

@t-mullen
Copy link
Member

t-mullen commented Sep 19, 2017

YJS uses a modified version of WOOT. I think a block-based approach like LogootSplit (which supports multicharacter inserts) would suit this project better. I'm working on it.

@kifhan
Copy link
Collaborator Author

kifhan commented Jul 28, 2018

Dotted LogootSplit: A delta-based sequence CRDT

I found this, have you checked?
To me, I can't tell... I need to see the working code sample to know this is actually working.

Anyway, It is cutting edge.
How do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants