​App cannot generate a card for a link

[kherge]

What happened?

1. Created a new page.
2. Created a new link block using /link.
3. Entered the URL (e.g. https://www.zillow.com/homedetails/339-Buckettown-Rd-Richmond-KY-40475/341933751_zpid/).
4. See an error message in the card.

Distribution version

macOS ARM 64 (Apple Silicon)

What browsers are you seeing the problem on if you're using web version?

No response

Are you self-hosting?

• Yes

Relevant log output

No response

Anything else?

No response

[affine-issue-bot]

Issue Status: ✅ Done

✅ Done

If the issue was a bug, it’s been fixed; if it was a missing feature, it’s been implemented.

^{_{This is an automatic reply by the bot.}}

[EYHN]

@darkskygit pls take a look

[darkskygit]

The firewall deployed by this website blocked our preview request.
We will adjust the behaviour of preview request later, but this may not have much effect.
We recommend that you contact the website to allow our preview API request.

[kherge]

I'm curious to know how Notion is capable of handling this correctly. Maybe the user agent request header is being spoofed? It is not realistic to request websites to make this change.

[darkskygit]

I'm curious to know how Notion is capable of handling this correctly. Maybe the user agent request header is being spoofed? It is not realistic to request websites to make this change.

They used a paid service, and the paid service uses AWS machines. Since the website you mentioned is also provided by AWS, I can assume that the service may has the right to bypass AWS WAF.

[kherge]

I know that Appflowy integrates third-party services (e.g. OpenAI), but requires the user to provide their own token to use it. Otherwise, it remains inactive. Could something similar be done for iframely? They seem to offer a good free tier that I could take advantage of.

[darkskygit]

I know that Appflowy integrates third-party services (e.g. OpenAI), but requires the user to provide their own token to use it. Otherwise, it remains inactive. Could something similar be done for iframely? They seem to offer a good free tier that I could take advantage of.

@fundon @fourdim what do you think about this? i think integrate into worker is more insensitive, or do you think it’s better to be integrated directly by bs?

[fundon]

@darkskygit
I would like to have a fallback button so that if the retrieval fails using a worker, the user can click on the fallback button to initiate a request from the client to retrieve the og meta.
Globally, user can also set to request resources directly from client.

[darkskygit]

@darkskygit I would like to have a fallback button so that if the retrieval fails using a worker, the user can click on the fallback button to initiate a request from the client to retrieve the . Globally, user can also set to request resources directly from client.og meta

browser site fetch may block by cors, this strategy only be used on electron

[fundon]

@darkskygit
I have tried to test the link-preview api, and received an error.

Toggle me!

λ curl 'http://127.0.0.1:8080/api/worker/link-preview' \
        -H 'Accept: */*' \
        -H 'Accept-Language: zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7' \
        -H 'Connection: keep-alive' \
        -H 'Content-Type: application/json' \
        -H 'Cookie: next-auth.csrf-token=917330990c359427fdb62955adb372482773d7c462b391f6e4e4053dfe2cc8a9%7C3fb90fd8afe3d5097552c502441ac6dba2d4c9d8254a72aef5617b9befb4a119; next-auth.callback-url=http%3A%2F%2Flocalhost%3A3010' \
        -H 'DNT: 1' \
        -H 'Origin: http://127.0.0.1:8080' \
        -H 'Referer: http://127.0.0.1:8080/workspace/nqemTGxEAjCk-knG-lCFq/sgSKARIlNxJOcnkcfVRdk' \
        -H 'Sec-Fetch-Dest: empty' \
        -H 'Sec-Fetch-Mode: cors' \
        -H 'Sec-Fetch-Site: same-origin' \
        -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
        -H 'sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"' \
        -H 'sec-ch-ua-mobile: ?0' \
        -H 'sec-ch-ua-platform: "macOS"' \
        --data-raw '{"url":"https://www.zillow.com/homedetails/339-Buckettown-Rd-Richmond-KY-40475/341933751_zpid/"}' -vv
*   Trying 127.0.0.1:8080...
* Connected to 127.0.0.1 (127.0.0.1) port 8080
> POST /api/worker/link-preview HTTP/1.1
> Host: 127.0.0.1:8080
> Accept: */*
> Accept-Language: zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7
> Connection: keep-alive
> Content-Type: application/json
> Cookie: next-auth.csrf-token=917330990c359427fdb62955adb372482773d7c462b391f6e4e4053dfe2cc8a9%7C3fb90fd8afe3d5097552c502441ac6dba2d4c9d8254a72aef5617b9befb4a119; next-auth.callback-url=http%3A%2F%2Flocalhost%3A3010
> DNT: 1
> Origin: http://127.0.0.1:8080
> Referer: http://127.0.0.1:8080/workspace/nqemTGxEAjCk-knG-lCFq/sgSKARIlNxJOcnkcfVRdk
> Sec-Fetch-Dest: empty
> Sec-Fetch-Mode: cors
> Sec-Fetch-Site: same-origin
> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
> sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"
> sec-ch-ua-mobile: ?0
> sec-ch-ua-platform: "macOS"
> Content-Length: 96
>
< HTTP/1.1 400 Bad Request
< X-Powered-By: Express
< date: Tue, 12 Mar 2024 11:23:29 GMT
< content-type: application/json;charset=UTF-8
< content-length: 24
< connection: close
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=bAdJY5lt6PQEZNz05q0MkbIhmU3FcG7M1kBP9x73nw5Kj5ZJkBuTZZC7VLNDfCcegoDPdezq0bw1JUO7Me0mSdkNzFdOEzQuRsi5kOc0p6Nd%2FWhjk6IbNm9AAyEcfg%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< strict-transport-security: max-age=15552000; includeSubDomains; preload
< x-content-type-options: nosniff
< server: cloudflare
< cf-ray: 86336995099a5316-LHR
< alt-svc: h3=":443"; ma=86400
< Vary: Accept-Encoding
<
* Closing connection
{"msg":"Invalid header"}

[fourdim]

Is it possible to use electron backend as a proxy?

[fundon]

We can try to adjust the header first, and reuse user-agent via web(extract from post-link-preview request).

curl -vvv https://www.zillow.com/homedetails/339-Buckettown-Rd-Richmond-KY-40475/341933751_zpid/ \
             -H 'Accept: */*' \
             -H 'Accept-Language: zh-CN,zh;q=0.9,zh-TW;q=0.8,en;q=0.7' \
             -H 'Connection: keep-alive' \
             -H 'DNT: 1' \
             -H 'Sec-Fetch-Dest: empty' \
             -H 'Sec-Fetch-Mode: cors' \
             -H 'Sec-Fetch-Site: same-origin' \
             -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \
             -H 'sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"' \
             -H 'sec-ch-ua-mobile: ?0' \
             -H 'sec-ch-ua-platform: "macOS"' | scraper 'meta'

[darkskygit]

i have updated the behavior of link preview api, now the website you provided is ready for preview