Added fresh parameter and readiness tests for EncryptedFS #58
Conversation
|
As discovered here, EFS is not passing a persistence test: MatrixAI/Polykey#266 (comment). This may have something to do with |
|
A variant of this is: If the That's super weird. However it seems strange we cannot get the underlying exception of |
|
Had to create an upstream issue about this error: Level/subleveldown#109 |
|
Managed to replicate the error. import DB from './src/DB';
async function main () {
const db = await DB.createDB({
dbPath: './tmp/db',
});
const mgrDomain = ['INodeManager'];
const dataDomain = [mgrDomain[0], 'data'];
const mgrDb = await db.level(mgrDomain[0]);
const dataDb = await db.level(dataDomain[1], mgrDb);
await db.put(['INodeManager', 'inodes', '123'], 'a', 'b');
await db.stop();
await db.start();
await db.level('123', dataDb);
console.log(await db.get(['INodeManager', 'inodes', '123'], 'a'));
await db.stop();
}
main();Upon calling |
|
I believe this would known as an uncaught rejection. Because this is not actually received by the catch handler of the Furthermore I realised that I don't have the types for Now at least I can see what kind of event types we can have. Maybe help get access to the open error. |
|
Through some sleuthing I found a way to actually acquire the error. It's quite sneaky. So basically During this construction, it will call the abstract level down constructor. Which is actually not in This ends up calling Notice that the This is the So it turns out that Now because the error is emitted on the constructed sublevel instance, that instance is what we need to watch for errors, not the root leveldb instance. dbLevelNew.on('error', (e) => {
console.log('GOT AN ERRROR!!!!!', e);
}); |
|
Upstream should indicate that |
|
The fix for the open problem is that the DB levels must be recreated upon This means This is good opportunity to just integrate |
|
The For this PR, I need to rework the Will need to update to 1.1.5 of |
|
Based on the idea of recreating the sublevels, my thinking is that just doing: // INodeManager needs to be recreated
// It contains sublevels that needs to be re-established
this.iNodeMgr = await INodeManager.createINodeManager({
db: this.db,
logger: this.logger.getChild(INodeManager.name),
fresh,
});In the However it doesn't seem to work. I'm still getting the |
|
If I recreate the sublevels right before I call Furthermore I noticed that Ok I just realised why recreating The problem is that this instance is also injected into It appears that therefore, we need to represent |
|
The One major change is that during creation/start, it doesn't gc dangling inodes. This was too difficult to do when CDSS and ready decorators are applied, because it could not call those destroy methods if the instance isn't yet started. However, instead the However 2 other tests failed: |
|
Appears that the source the error comes from It thinks that it's no longer running... need to investigate why. |
|
Ok it turns out the error is the same problem with trying to do GC operations in Basically the current async-init will say that it's not ready if we are in the middle of stopping because it checking the lock now. Therefore we do have to do some code duplication and create a separate |
|
Tests are passing now. Now finalising the linting. |
Both the EncryptedFS and INodeManager is now following the CDSS pattern
|
Ok it's all done. Time to merge. |
Description
The EFS didn't have a
freshparameter. Adding this in to match MatrixAI/js-db#6.Also noticed that EFS doesn't have readiness testing. I'm adding this in to
tests/EncryptedFS.test.ts.Related to MatrixAI/Polykey#266 (comment)
Tasks
freshtoEncryptedFSand maybeINodeManagersubleveldown()and listening for open errors whensubleveldownis calledFinal checklist