Converting from LizardFS

[smurfix]

Have you read through available documentation and open Github issues?

yes

Is this a BUG report, FEATURE request, or a QUESTION? Who is the indended audience?

Feature

The problem

I currently have a large-ish LizardFS installation. Development of LizardFS appears to have stalled and there are other problems; most importantly, I need to move the master from amd64 to arm64; attempting to do that reliably crashes the new master.

Thus I'd like to migrate to MooseFS, preferably without copying far too many terabytes of data for no good reason.

As the binary formats of the master log are not compatible (obviously), the way forward that would benefit the most people IMHO is a mfsmeta_un_dump that reads the textual output from mfsmetadump and converts it back to a binary file. (Mangling their output to look like yours should be reasonably easy.)

I could write that myself if I had a concise documentation of the binary file format. The code from mfsmetadump.c isn't quite sufficient …

I also didn't look at whether the chunkservers need any changes, that'd be the next step.

[chogata]

There are many issues with your plan:

1. There are most probably changes in the chunk format (we changed it since the fork happend and they probably changed something too), so Lizard chunks will not be compatible with MooseFS chunks.
2. Metadata is not compatible either, so even if you got a text dump of Lizard metadata and tried to convert it to a text dump to MooseFS metadata, you would probably have to sacrifice a lot of information. Basically, you would only be able to convert edges (directories) and leaves (files) with basic info like gid/uid, nothitng else: no extra attributes, no labels (the concept works very differently on those two systems).
3. This would be a delicate operation, one that begs for a backup. And if you were to back up your data, the most obvious solution would be to just copy it to another system...

[Blackpaw]

Thanks for this, I just asked a similar question on Q&A. Fortunately I have enough space and data to do a "in place" migration.

[borkd]

If you were using plain replication goals your best bet and fastest route is migration in place. This assumes you have enough existing resources to stand up a separate MooseFS master and metalogger and can start MooseFS chunkservers alongside LizardFS chunkservers, but on different port. Reduce trash time in LizardFS to the shortest acceptable window and start moving files from LizardFS mountpoint to the MooseFS mountpoint. Check your hardware before you start and make sure you don't run out of disk space (combination of available space and trash time)

[4Dolio]

This is also on my to-do list. Probably end up doing it all the hard way, one chunk of data at a time.