Stuck on indexing

[luandro]

Branch master was working fine until this morning. It suddenly started indexing and gets stuck. It logs a bunch of errors are soon as I start it:

Error {}
Error {}
Error {}
Error {}
RangeError: Maximum call stack size exceeded
    at module.exports (/Users/patchwork/node_modules/ssb-server/node_modules/explain-error/index.js:25:18)
    at /Users/patchwork/node_modules/ssb-server/node_modules/flumedb/index.js:139:35
    at /Users/patchwork/node_modules/pull-stream/sinks/drain.js:20:24
    at /Users/patchwork/node_modules/ssb-server/node_modules/pull-stream/throughs/async-map.js:28:19
    at /Users/patchwork/node_modules/ssb-server/node_modules/pull-cursor/skip.js:12:26
    at /Users/patchwork/node_modules/ssb-server/node_modules/pull-cursor/cursor.js:28:28
    at /Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/inject.js:42:22
    at /Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:48:11
    at Object.read (/Users/patchwork/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:49:14)
    at next (/Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:47:16)
error trying to reach github to check for latest patchwork version:  Error {
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '192.30.253.116',
  port: 443 }
Error: when trying to retrive:1557915534567at since:697735799

Doesn't log anything else anymore after that.

I've tried deleting my gossip.json, using other secret and opening other clients, but it's always stuck on indexing.

Any ideas?

[christianbundy]

Hey @luandro, thanks for the bug report.

Could you try installing with npm ci and test whether that resolves the problem? If that doesn't work, I'd be curious to know which commit this problem started with.

[christianbundy]

That looks to me like one of your flumelog-level instances became corrupted. I'd recommend deleting all of the directories (not the files!) in ~/.ssb/flume and then restarting your Scuttlebutt client.

Here's a command that should let you interactively delete those directories -- I think it's safe to say y and confirm deletion on all of the results, but [as always] it's best to double-check and make sure it doesn't try to delete log.offset or something silly.

You'll want to run:

find ~/.ssb/flume/ -mindepth 1 -maxdepth 1 -type d -ok rm -rf {} \;

For me, that looks like:

$ find ~/.ssb/flume/ -mindepth 1 -maxdepth 1 -type d -ok rm -rf {} \;
< rm ... /home/christianbundy/.ssb/flume/search > ? y
< rm ... /home/christianbundy/.ssb/flume/feed > ? y
< rm ... /home/christianbundy/.ssb/flume/query > ? y
< rm ... /home/christianbundy/.ssb/flume/private--oaWWDs8g > ? y
< rm ... /home/christianbundy/.ssb/flume/backlinks--oaWWDs8g > ? y
< rm ... /home/christianbundy/.ssb/flume/time > ? y
< rm ... /home/christianbundy/.ssb/flume/links > ? y
< rm ... /home/christianbundy/.ssb/flume/clock > ? y

[luandro]

Got carried away with a bunch of things, sorry. Tested running npm ci and in returns an error.

Cleaning all the flume folders didn't do the trick either. But I'll do some better tests tomorrow.

[christianbundy]

No problem, thanks for all the time you're spending testing and debugging! ❤️ I know none of this is very fun.

[luandro]

Heya @christianbundy. Back again trying to get SSB working. Last time this happened I had to delete everything but my secret, any clues on what other files/folders could solve the issue?

I renamed the gossip.json file and it seems a new one isn't being generated.

[christianbundy]

Hey! Are you still seeing this?

Error: when trying to retrive:1557915534567at since:697735799

That should have gone away when you deleted those files above, so I'm confused at why that might still be showing up.

[luandro]

You're right, that doesn't seem to appear any more.

But I get stuck on RangeError: Maximum call stack size exceeded and a bunch of Error {} messages.

[christianbundy]

Does it still have that error stack included? I imagine it's different now?

[luandro]

Last message before the errors is that git-ssb isn't installed, but I wouldn't think that could be the problem.

This is the only erro I get:

RangeError: Maximum call stack size exceeded
    at /Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/inject.js:42:22
    at /Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:48:11
    at Object.read (/Users/patchwork/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:49:14)
    at next (/Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:47:16)
    at /Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:43:11
    at /Users/patchwork/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:96:11
    at get (/Users/patchwork/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:29:7)
    at Object.readUInt32BE (/Users/patchwork/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:93:9)
    at /Users/patchwork/node_modules/ssb-server/node_modules/flumelog-offset/frame/recoverable.js:41:16
    at /Users/patchwork/node_modules/ssb-server/node_modules/aligned-block-file/blocks.js:96:11

[christianbundy]

Hmm. My guess is that it's a corrupt ~/.ssb/flume/log.offset, but I'm not sure. The quickfix would be to delete it but that doesn't mean the problem is actually solved. 😕

[luandro]

Renamed it, gonna take a while to re-populated it. Let's see if after the sync it works again.

[luandro]

Deleting it did the trick, guess I'll close this for now since it's nothing that could be fixed in code. Thanks @christianbundy.