Followers disappearing after upgrade to 6.0 #6131

FediVideos · 2023-12-20T18:39:13Z

Describe the current behavior

I'm posting on behalf of instance owner https://peertube.lon.tv/a/lonseidman/video-channels, they have recently upgraded to PeerTube 6.0 and their followers started disappearing. They were on around 100 followers or so before the upgrade?

Steps to reproduce

(Not applicable)

Describe the expected behavior

The follower numbers should only drop if people unfollow. As far as I can tell, people have not been unfollowing.

Additional information

PeerTube instance: peertube.lon.tv
- URL: https://peertube.lon.tv
- Version: 6.0.2
- NodeJS version:
- Ffmpeg version:
Browser name, version and platforms on which you could reproduce the bug:
Link to browser console log if relevant:
Link to server log if relevant (journalctl or /var/www/peertube/storage/logs/):

I can request logs from owner if it would be useful?

The text was updated successfully, but these errors were encountered:

kodxana · 2023-12-20T21:54:11Z

From what I see from my instance it look like lon.tv has some issues

peert-peertube-1  |     "stack": "Error: Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube\n    at getOrCreateAPActor (file:///app/dist/core/lib/activitypub/actors/get.js:18:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async addVideoShare (file:///app/dist/core/lib/activitypub/share.js:43:19)\n    at async Bluebird.map.concurrency (file:///app/dist/core/lib/activitypub/share.js:27:13)",
peert-peertube-1  |     "message": "Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube"
peert-peertube-1  |   }
peert-peertube-1  | }

I would check if private/public keys are there as I had similar errors after upgrade to 6.0

Check database for missing keys:

select * from actor where "preferredUsername" = 'peertube' and "serverId" is null;

If there are missing privkeys you want generate them

create js script example script.js

const options = {
  modulusLength: 2048,
  publicKeyEncoding: {
    type: 'spki',
    format: 'pem'
  },
  privateKeyEncoding: {
    type: 'pkcs1',
    format: 'pem'
  }
}

require('crypto').generateKeyPair('rsa', options, (err, publicKey, privateKey) => {
  if (err) return console.error(err)

  console.log('Public key')
  console.log(publicKey)
  console.log('Private key')
  console.log(privateKey)
})

After you run this it should give priv pub keys

Add key back to database

UPDATE actor
SET "publicKey" = 'your_public_key_string', "privateKey" = 'your_private_key_string'
WHERE id = 1;

That should fix federation.

Chocobozzz · 2023-12-21T07:11:13Z

Hi,

@kodwana is right: we can't follow the [peertube.lon.tv](https://peertube.lon.tv/) instance and the database manipulation should fix that.

But I think there's a different issue there, because https://peertube.lon.tv/a/lonseidman account doesn't have a missing public key. It may be related to #6113 and the upgrade to Node 18, where PeerTube can't do requests to followers and so remove them after multiple request failures.

Can you ask the admin to provide some logs?

FediVideos · 2023-12-21T15:50:03Z

Thanks both for answers, I've messaged the instance owner and asked if they can supply logs.

lonseidman · 2023-12-22T20:46:35Z

Hi everyone thanks for offering to help figure this out. I am happy to provide logs, just let me know which logs you're looking for.

Also of note - I am running Peertube via Docker.

-Lon

kodxana · 2023-12-22T22:32:29Z

Hi everyone thanks for offering to help figure this out. I am happy to provide logs, just let me know which logs you're looking for.

Also of note - I am running Peertube via Docker.

-Lon

I also run via Docker.
Btw you can reach me via Matrix if you want to: @Madiator2011:matrix.madiator.dev

Chocobozzz · 2023-12-26T10:26:37Z

Hi everyone thanks for offering to help figure this out. I am happy to provide logs, just let me know which logs you're looking for.

Please paste warn logs from Administration -> System -> Logs (select Standard Logs).

You can also paste failed Local Jobs error (activitypub-http-broadcast or activitypub-http-unicast jobs for example) to see what is the root cause of the federation problem

lonseidman · 2023-12-27T13:16:52Z

I have a bunch of these, looks like there's a key it's missing:

warn[12/27/2023, 8:13:05 AM] Cannot process activity View. { "err": { "stack": "RequestError: options.key must be a sshpk.PrivateKey\n at Request._beforeError (file:///app/node_modules/got/dist/source/core/index.js:310:21)\n at Request.flush (file:///app/node_modules/got/dist/source/core/index.js:299:18)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at Object.signRequest (/app/node_modules/@peertube/http-signature/lib/signer.js:362:14)\n at got.extend.Object.assign.hooks.beforeRequest (file:///app/dist/core/helpers/requests.js:49:35)\n at Request._makeRequest (file:///app/node_modules/got/dist/source/core/index.js:942:34)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Request.flush (file:///app/node_modules/got/dist/source/core/index.js:285:13)", "message": "options.key must be a sshpk.PrivateKey", "name": "RequestError", "requestHeaders": { "user-agent": "PeerTube/6.0.2 (+https://peertube.lon.tv)", "date": "Wed, 27 Dec 2023 13:13:05 GMT", "accept": "application/activity+json, application/ld+json", "accept-encoding": "gzip, deflate, br", "host": "neat.tube" } } }

kodxana · 2023-12-27T13:18:37Z

I have a bunch of these, looks like there's a key it's missing:

`warn[12/27/2023, 8:13:05 AM] Cannot process activity View.

{

"err": {

"stack": "RequestError: options.key must be a sshpk.PrivateKey\n    at Request._beforeError (file:///app/node_modules/got/dist/source/core/index.js:310:21)\n    at Request.flush (file:///app/node_modules/got/dist/source/core/index.js:299:18)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at Object.signRequest (/app/node_modules/@peertube/http-signature/lib/signer.js:362:14)\n    at got.extend.Object.assign.hooks.beforeRequest (file:///app/dist/core/helpers/requests.js:49:35)\n    at Request._makeRequest (file:///app/node_modules/got/dist/source/core/index.js:942:34)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Request.flush (file:///app/node_modules/got/dist/source/core/index.js:285:13)",

"message": "options.key must be a sshpk.PrivateKey",

"name": "RequestError",

"requestHeaders": {

  "user-agent": "PeerTube/6.0.2 (+https://peertube.lon.tv)",

  "date": "Wed, 27 Dec 2023 13:13:05 GMT",

  "accept": "application/activity+json, application/ld+json",

  "accept-encoding": "gzip, deflate, br",

  "host": "neat.tube"

}

}

}`

Yes that what i suspected you got private key issue one I got.

kodxana · 2023-12-27T13:39:30Z

@lonseidman so if you run peertube inside docker you need to ssh into pogester container for your instance

docker exec -it [name of your postgres docker container] /bin/bash
In my case I had it

Once you get terminal from container you need to open psql tool:

psql -d DATABASENAME -U username
example: psql -d peertube -U madiator (this one usually is set in dockerf-compose file)

Then I recomend sending this command first (it makes output more readable in terminal
\x

You will get infor that Expanded display is on

Then you want to check for channels with missing private public keys:

select * from actor where "preferredUsername" = 'peertube' and "serverId" is null;

You might want to save id for channel without keys.

If any channel has key missing you need to generate priv public key

with the script I have shared
#6131 (comment)
just make sure to set correct user ID

WHERE id = 1;

kodxana · 2024-01-11T09:27:29Z

@lonseidman Looks like there are still issues :)

{
  "payload": {
    "host": "peertube.lon.tv:443",
    "name": "peertube",
    "followerActorId": 1
  },
  "err": {
    "stack": "Error: Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube\n    at getOrCreateAPActor (file:///app/dist/core/lib/activitypub/actors/get.js:18:19)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Object.processActivityPubFollow [as activitypub-follow] (file:///app/dist/core/lib/job-queue/handlers/activitypub-follow.js:23:23)\n    at async Object.wrapPromiseFun (file:///app/dist/core/lib/plugins/hooks.js:8:24)\n    at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:350:28)\n    at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:535:24)",
    "message": "Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube"
  }
}

lonseidman · 2024-01-11T15:17:56Z

Yeah I haven't had time to address this.. Hopefully next week.

…

On Thu, Jan 11, 2024 at 1:27 AM kodxana ***@***.***> wrote: @lonseidman <https://github.com/lonseidman> Looks like there are still issues :) { "payload": { "host": "peertube.lon.tv:443", "name": "peertube", "followerActorId": 1 }, "err": { "stack": "Error: Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube\n at getOrCreateAPActor (file:///app/dist/core/lib/activitypub/actors/get.js:18:19)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Object.processActivityPubFollow [as activitypub-follow] (file:///app/dist/core/lib/job-queue/handlers/activitypub-follow.js:23:23)\n at async Object.wrapPromiseFun (file:///app/dist/core/lib/plugins/hooks.js:8:24)\n at async Worker.processJob (/app/node_modules/bullmq/dist/cjs/classes/worker.js:350:28)\n at async Worker.retryIfFailed (/app/node_modules/bullmq/dist/cjs/classes/worker.js:535:24)", "message": "Cannot fetch remote actor https://peertube.lon.tv/accounts/peertube" } } — Reply to this email directly, view it on GitHub <#6131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AORPSHE6TVXLM7WVW3ZI7FLYN6WAZAVCNFSM6AAAAABA5II33OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBWG4YDMMJXHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- ---------- Lon Seidman 860-577-0566 ***@***.***

lonseidman · 2024-01-27T19:48:51Z

So I finally had some time to tackle this - as expected the keys were missing from the database record which I generated and inserted. I've recycled the Docker containers too - hopefully this rectifies it!

kodxana · 2024-01-28T11:13:10Z

Make sure to check if keys are there if not you need to generate them manually

lonseidman · 2024-01-28T16:05:11Z

Yes I generated them - if you can test and see if we're able to connect properly that would be helpful!

Chocobozzz added the Status: Waiting for answer Waiting issue author answer label Dec 21, 2023

Chocobozzz closed this as completed Jan 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Followers disappearing after upgrade to 6.0 #6131

Followers disappearing after upgrade to 6.0 #6131

FediVideos commented Dec 20, 2023 •

edited

Loading

kodxana commented Dec 20, 2023 •

edited

Loading

Chocobozzz commented Dec 21, 2023 •

edited

Loading

FediVideos commented Dec 21, 2023

lonseidman commented Dec 22, 2023

kodxana commented Dec 22, 2023

Chocobozzz commented Dec 26, 2023

lonseidman commented Dec 27, 2023

kodxana commented Dec 27, 2023

kodxana commented Dec 27, 2023

kodxana commented Jan 11, 2024

lonseidman commented Jan 11, 2024 via email

lonseidman commented Jan 27, 2024

kodxana commented Jan 28, 2024

lonseidman commented Jan 28, 2024

Followers disappearing after upgrade to 6.0 #6131

Followers disappearing after upgrade to 6.0 #6131

Comments

FediVideos commented Dec 20, 2023 • edited Loading

Describe the current behavior

Steps to reproduce

Describe the expected behavior

Additional information

kodxana commented Dec 20, 2023 • edited Loading

Chocobozzz commented Dec 21, 2023 • edited Loading

FediVideos commented Dec 21, 2023

lonseidman commented Dec 22, 2023

kodxana commented Dec 22, 2023

Chocobozzz commented Dec 26, 2023

lonseidman commented Dec 27, 2023

kodxana commented Dec 27, 2023

kodxana commented Dec 27, 2023

kodxana commented Jan 11, 2024

lonseidman commented Jan 11, 2024 via email

lonseidman commented Jan 27, 2024

kodxana commented Jan 28, 2024

lonseidman commented Jan 28, 2024

FediVideos commented Dec 20, 2023 •

edited

Loading

kodxana commented Dec 20, 2023 •

edited

Loading

Chocobozzz commented Dec 21, 2023 •

edited

Loading