Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal Server Error when trying to resolve Actor #72

Closed
allouis opened this issue Jun 18, 2024 · 10 comments
Closed

Internal Server Error when trying to resolve Actor #72

allouis opened this issue Jun 18, 2024 · 10 comments
Labels
bug Something isn't working

Comments

@allouis
Copy link
Contributor

allouis commented Jun 18, 2024

I've configured the Federation instance to have an actor and keypair dispatcher like below (Currently I'm returning null for debugging purposes, but the error happens when I return a keypair, too)

fedify.setActorDispatcher("/activitypub/users/{handle}", async (ctx, handle) => {
    console.log(`actorDispatcher(${handle})`)
    if (handle !== "index") return null;

    const data = await getUserData(ctx, handle);
    return new Person(data);
}).setKeyPairDispatcher(async (_ctx, handle) => {
    console.log(`keypairDispatcher(${handle})`)
    if (handle !== "index") return null;
    return null;
});

When I try to read the Actor data via curl like:

curl -H 'Accept: application/activity+json' http://site.com/activitypub/users/index 

I get this error:

activitypub-1  | 04:41:14.659 DBG fedify·runtime·docloader Fetching document: 'GET' 'https://www.w3.org/ns/activitystreams' { accept: 'application/activity+json, application/ld+json' }
activitypub-1  | 04:41:15.319 DBG fedify·runtime·docloader Fetched document: 200 'https://www.w3.org/ns/activitystreams' {
activitypub-1  |   'access-control-allow-origin': '*',
activitypub-1  |   'alt-svc': 'h3=":443"; ma=86400',
activitypub-1  |   'cache-control': 'max-age=21600',
activitypub-1  |   'cf-cache-status': 'BYPASS',
activitypub-1  |   'cf-ray': '89589b1c5902c8f3-BKK',
activitypub-1  |   connection: 'keep-alive',
activitypub-1  |   'content-encoding': 'br',
activitypub-1  |   'content-location': 'activitystreams.jsonld',
activitypub-1  |   'content-security-policy': "frame-ancestors 'self' https://cms.w3.org/ https://cms-dev.w3.org/; upgrade-insecure-requests",
activitypub-1  |   'content-type': 'application/ld+json',
activitypub-1  |   date: 'Tue, 18 Jun 2024 04:41:15 GMT',
activitypub-1  |   etag: 'W/"1f31-5b3aa97d9d140;d7-60e7311355640',
activitypub-1  |   expires: 'Tue, 18 Jun 2024 10:41:15 GMT',
activitypub-1  |   'last-modified': 'Mon, 09 Nov 2020 11:09:17 GMT',
activitypub-1  |   server: 'cloudflare',
activitypub-1  |   'set-cookie': '__cf_bm=6d0lVaf1RaT_2SUXqlcisS4VA3A3KqwQIVNIvxUNoGg-1718685675-1.0.1.1-Ht0iamouvN6u5hiUNe.MyIcK7y3XzB1I3oL6pllz_GpRQBVh0MkER9J1WSJVVzqVilV6YS2.7q.TTIh1jeHWww; path=/; expires=Tue, 18-Jun-24 05:11:15 GMT; domain=.w3.org; HttpOnly; Secure; SameSite=None',
activitypub-1  |   'strict-transport-security': 'max-age=15552000; includeSubdomains; preload',
activitypub-1  |   tcn: 'choice',
activitypub-1  |   'transfer-encoding': 'chunked',
activitypub-1  |   vary: 'negotiate,accept, Accept-Encoding',
activitypub-1  |   'x-backend': 'www-mirrors',
activitypub-1  |   'x-request-id': '89589b1c5902c8f3'
activitypub-1  | }
activitypub-1  | 04:41:15.332 DBG fedify·runtime·docloader Fetching document: 'GET' 'https://w3id.org/security/v1' { accept: 'application/activity+json, application/ld+json' }
activitypub-1  | jsonld.InvalidUrl: Dereferencing a URL did not result in a valid JSON-LD object. Possible causes are an inaccessible URL perhaps due to a same-origin policy (ensure the server uses CORS if you are using client-side JavaScript), too many redirects, a non-JSON response, or more than one HTTP Link Header was provided for a remote context.
activitypub-1  |     at ContextResolver._fetchContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:173:13)
activitypub-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
activitypub-1  |     at async ContextResolver._resolveRemoteContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:117:34)
activitypub-1  |     at async ContextResolver.resolve (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:50:22)
activitypub-1  |     at async api.process (/opt/activitypub/node_modules/jsonld/lib/context.js:87:20)
activitypub-1  |     at async jsonld.compact (/opt/activitypub/node_modules/jsonld/lib/jsonld.js:176:21)
activitypub-1  |     at async Person.toJsonLd (file:///opt/activitypub/node_modules/@fedify/fedify/esm/vocab/vocab.js:13216:16)
activitypub-1  |     at async handleActor (file:///opt/activitypub/node_modules/@fedify/fedify/esm/federation/handler.js:34:20)
activitypub-1  |     at async #fetch (file:///opt/activitypub/node_modules/@fedify/fedify/esm/federation/middleware.js:939:24)
activitypub-1  |     at async Federation.fetch (file:///opt/activitypub/node_modules/@fedify/fedify/esm/federation/middleware.js:895:26) {
activitypub-1  |   details: {
activitypub-1  |     code: 'loading remote context failed',
activitypub-1  |     url: 'https://w3id.org/security/v1',
activitypub-1  |     cause: TypeError: fetch failed
activitypub-1  |         at node:internal/deps/undici/undici:12502:13
activitypub-1  |         at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
activitypub-1  |         at async fetchDocumentLoader (file:///opt/activitypub/node_modules/@fedify/fedify/esm/runtime/docloader.js:70:22)
activitypub-1  |         at async file:///opt/activitypub/node_modules/@fedify/fedify/esm/runtime/docloader.js:161:31
activitypub-1  |         at async ContextResolver._fetchContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:166:19)
activitypub-1  |         at async ContextResolver._resolveRemoteContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:117:34)
activitypub-1  |         at async ContextResolver.resolve (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:50:22)
activitypub-1  |         at async api.process (/opt/activitypub/node_modules/jsonld/lib/context.js:87:20)
activitypub-1  |         at async jsonld.compact (/opt/activitypub/node_modules/jsonld/lib/jsonld.js:176:21)
activitypub-1  |         at async Person.toJsonLd (file:///opt/activitypub/node_modules/@fedify/fedify/esm/vocab/vocab.js:13216:16) {
activitypub-1  |       [cause]: [AggregateError]
activitypub-1  |     }
activitypub-1  |   }
activitypub-1  | }

I then updated to only set an actor dispatcher like so:

fedify.setActorDispatcher("/activitypub/users/{handle}", async (ctx, handle) => {
    console.log(`actorDispatcher(${handle})`)
    if (handle !== "index") return null;

    const data = await getUserData(ctx, handle);
    return new Person(data);
});

And curling this gives the same error!

@dahlia
Copy link
Owner

dahlia commented Jun 18, 2024

Please let me know the version of Fedify and Node.js you are using.

@allouis
Copy link
Contributor Author

allouis commented Jun 18, 2024

@dahlia We're using fedify@0.9.1 and node v20.14.0

@allouis
Copy link
Contributor Author

allouis commented Jun 18, 2024

I think this issue is related: digitalbazaar/jsonld.js#451

Specifically this comment digitalbazaar/jsonld.js#451 (comment)

What is the reasoning behind using "manual" mode for redirects? Is that part of the JSON-LD spec for resolving contexts?

@allouis
Copy link
Contributor Author

allouis commented Jun 18, 2024

This seems to not be an issue with Fedify, this stems from using Docker somehow...

Both of these commands should not error

> node -e "fetch('https://w3id.org/security/v1').then(console.log, console.log)"
> node -e "fetch('https://www.w3.org/ns/activitystreams').then(console.log, console.log)"

But, if I run these commands inside of docker - the first one will error

If I run:

docker run -it node:lts-alpine node -e "fetch('https://w3id.org/security/v1').then(console.log, console.log)"

Then I get:

TypeError: fetch failed
    at node:internal/deps/undici/undici:12502:13
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  [cause]: AggregateError [ETIMEDOUT]: 
      at internalConnectMultiple (node:net:1117:18)
      at internalConnectMultiple (node:net:1185:5)
      at Timeout.internalConnectMultipleTimeout (node:net:1711:5)
      at listOnTimeout (node:internal/timers:575:11)
      at process.processTimers (node:internal/timers:514:7) {
    code: 'ETIMEDOUT',
    [errors]: [ [Error], [Error] ]
  }
}

This is really weird behaviour and can be replicated by some members of our team, but not all!

Is there an easy way to hardcode the values of this into the cache?

@dahlia
Copy link
Owner

dahlia commented Jun 18, 2024

That's indeed very strange!

Is there an easy way to hardcode the values of this into the cache?

The best way would be to implement your own context loader! E.g.,

import { fetchDocumentLoader, RemoteDocument } from "@fedify/fedify";

async function cachedContextLoader(url: string): Promise<RemoteDocument> {
  if (url === "https://w3id.org/security/v1") {
    return {
      contextUrl: null,
      documentUrl: url,
      document: {
        "@context": { ... }
      }
    };
  }
  return await fetchDocumentLoader(url);
}

const federation = new Federation<void>({
  contextLoader: cachedContextLoader,
});

@dahlia dahlia added the bug Something isn't working label Jun 18, 2024
@allouis
Copy link
Contributor Author

allouis commented Jun 19, 2024

Hey @dahlia

I've tried your suggestion but I'm still running into issues

Here's what I've tried:

const securityDoc = {
    contextUrl: null,
    documentUrl: 'https://w3id.org/security/v1',
    document: {
        "@context": {
          ...
        }
    }
};

import { fetchDocumentLoader, RemoteDocument, getAuthenticatedDocumentLoader } from "@fedify/fedify";

async function cachedContextLoader(url: string): Promise<RemoteDocument> {
    if (url === "https://w3id.org/security/v1" || url === "https://w3id.org/security/v1/") {
      return securityDoc;
  }
  return await fetchDocumentLoader(url);
}

const fedifyKv = new MemoryKvStore();

fedifyKv.set(['_fedify', 'remoteDocument', 'https://w3id.org/security/v1/'], securityDoc);

fedifyKv.set(['_fedify', 'remoteDocument', 'https://w3id.org/security/v1'], securityDoc);

const fedify = new Federation<ContextData>({
    kv: fedifyKv,
    contextLoader: cachedContextLoader,
    documentLoader: cachedContextLoader,
    authenticatedDocumentLoaderFactory: (identity) => {
        const loader = getAuthenticatedDocumentLoader(identity);

        return async function load(url) {
            if (url === "https://w3id.org/security/v1" || url === "https://w3id.org/security/v1/") {
                return securityDoc;
            }
            return await loader(url);
        }
    },
    treatHttps: true
});

And I am getting this error when attempting to send activities to other actors

activitypub-1  | 02:58:52.009 DBG fedify·runtime·docloader Fetching document: 'GET' 'https://w3id.org/security/v1' { accept: 'application/activity+json, application/ld+json' }
activitypub-1  | jsonld.InvalidUrl: Dereferencing a URL did not result in a valid JSON-LD object. Possible causes are an inaccessible URL perhaps due to a same-origin policy (ensure the server uses CORS if you are using client-side JavaScript), too many redirects, a non-JSON response, or more than one HTTP Link Header was provided for a remote context.
activitypub-1  |     at ContextResolver._fetchContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:173:13)
activitypub-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
activitypub-1  |     at async ContextResolver._resolveRemoteContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:117:34)
activitypub-1  |     at async ContextResolver.resolve (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:50:22)
activitypub-1  |     at async api.process (/opt/activitypub/node_modules/jsonld/lib/context.js:87:20)
activitypub-1  |     at async api.expand (/opt/activitypub/node_modules/jsonld/lib/expand.js:214:17)
activitypub-1  |     at async jsonld.expand (/opt/activitypub/node_modules/jsonld/lib/jsonld.js:322:18)
activitypub-1  |     at async Object.fromJsonLd (file:///opt/activitypub/node_modules/@fedify/fedify/esm/vocab/vocab.js:1993:30)
activitypub-1  |     at async lookupObject (file:///opt/activitypub/node_modules/@fedify/fedify/esm/vocab/lookup.js:77:16)
activitypub-1  |     at <anonymous> (/opt/activitypub/src/app.ts:302:27) {
activitypub-1  |   details: {
activitypub-1  |     code: 'loading remote context failed',
activitypub-1  |     url: 'https://w3id.org/security/v1',
activitypub-1  |     cause: TypeError: fetch failed
activitypub-1  |         at node:internal/deps/undici/undici:12502:13
activitypub-1  |         at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
activitypub-1  |         at async fetchDocumentLoader (file:///opt/activitypub/node_modules/@fedify/fedify/esm/runtime/docloader.js:70:22)
activitypub-1  |         at async ContextResolver._fetchContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:166:19)
activitypub-1  |         at async ContextResolver._resolveRemoteContext (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:117:34)
activitypub-1  |         at async ContextResolver.resolve (/opt/activitypub/node_modules/jsonld/lib/ContextResolver.js:50:22)
activitypub-1  |         at async api.process (/opt/activitypub/node_modules/jsonld/lib/context.js:87:20)
activitypub-1  |         at async api.expand (/opt/activitypub/node_modules/jsonld/lib/expand.js:214:17)
activitypub-1  |         at async jsonld.expand (/opt/activitypub/node_modules/jsonld/lib/jsonld.js:322:18)
activitypub-1  |         at async Object.fromJsonLd (file:///opt/activitypub/node_modules/@fedify/fedify/esm/vocab/vocab.js:1993:30) {
activitypub-1  |       [cause]: [AggregateError]
activitypub-1  |     }
activitypub-1  |   }
activitypub-1  | }

It's as if the context isn't always loaded through the document/context loader and is falling back to the default? Do you know what could be causing this?

@dahlia
Copy link
Owner

dahlia commented Jun 19, 2024

Looks like you've used lookupObject() from your app, right? Could you try to pass your custom context loader to lookupObject() too? It should work!

@dahlia
Copy link
Owner

dahlia commented Jun 19, 2024

@allouis By the way, I filed a new issue you probably will be interested in: #74.

@dahlia
Copy link
Owner

dahlia commented Jun 26, 2024

@allouis Is your issue now resolved? Would it be okay if I close this issue?

@allouis
Copy link
Contributor Author

allouis commented Jun 27, 2024

Sorry I missed this - yeah this is resolved now with the hardcoded contexts!

@allouis allouis closed this as completed Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants