Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use worker bindings interacting from api to gateway #136

Merged
merged 2 commits into from Jun 30, 2022

Conversation

vasco-santos
Copy link
Member

@vasco-santos vasco-santos commented Jun 2, 2022

This PR adds:

  • worker bindings from API to Gateway

Implementation details:

  • miniflare support for testing bindings is still quite limited, we cannot spin up the other worker build, and it does not seem to support ESM either...
  • bindings from API to Gateway tests use custom script that does the fetch to the local daemon converting subdomain resolution to path resolution. This is because we can't resolve localhost subdomains locally unless we add them to /etc/hosts, which is not cool, given we can't use tools like https://github.com/feross/hostile without sudo for pre and post tests... There was the option of relying on services like https://readme.localtest.me/ but don't really want to use external services in CI

Needs:

  • wrangler 2

Initial binding part of #101 to move to worker bindings setup

@cloudflare-pages
Copy link

cloudflare-pages bot commented Jun 2, 2022

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 762c382
Status: ✅  Deploy successful!
Preview URL: https://f457bde5.nftstorage-link.pages.dev
Branch Preview URL: https://fix-use-worker-bindings.nftstorage-link.pages.dev

View logs

@vasco-santos vasco-santos force-pushed the fix/use-worker-bindings branch 3 times, most recently from 01390ed to e1ea575 Compare June 2, 2022 19:27
@vasco-santos vasco-santos marked this pull request as ready for review June 2, 2022 20:29
@vasco-santos vasco-santos mentioned this pull request Jun 3, 2022
2 tasks
@vasco-santos vasco-santos changed the title fix: use worker bindings feat: use worker bindings interacting from api to gateway Jun 3, 2022
@vasco-santos vasco-santos mentioned this pull request Jun 3, 2022
2 tasks
@olizilla olizilla requested review from olizilla and removed request for alanshaw June 30, 2022 10:03
Copy link
Contributor

@olizilla olizilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code changes look good, but I need a bit more context on what you know so far about this "app-specific-gateway worker to shared-gateway worker" pattern will get us.

Will it really get us a common cache? Or do we need to test it to find out?

If a bound worker is really just code bundled with another worker, I would guess that we still wouldn't benefit from a shared common cache, but each app-gateway-specific-gateway would just be writiing to it's own cache just using the shared code of the shared-gateway.


[[env.production.r2_buckets]]
bucket_name = "super-hot"
binding = "SUPERHOT"

[[env.production.unsafe.bindings]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are unsafe.bindings documented anywhere? Can you link to docs or add a note here about how they work and why they may or may not be unsafe?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All i can find is this PR which mentions they are undocumented cloudflare/workers-sdk#1004

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and this... the list of drawbacks listed on this PR is scary
cloudflare/workers-sdk#906

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unsafe is not needed anymore in fact. They already support without it :)

The drawbacks there seem outdated by know from what I have seen in their discord

* @param {URL} url
*/
function getCidFromSubdomainUrl(url) {
// Replace "ipfs-staging" by "ipfs" if needed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment looks like a copy-pasta... or is there a deployment issue i'm not seeing

Suggested change
// Replace "ipfs-staging" by "ipfs" if needed

[[env.production.unsafe.bindings]]
name = "GATEWAY"
type = "service"
service = "gateway-nft-storage-production"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have a naming scheme for our services? Should we define one?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably should re-think this, but perhaps as part of the new workers read architecture. This name is the name of the already deployed nftstorage.link worker

@vasco-santos
Copy link
Member Author

vasco-santos commented Jun 30, 2022

Will it really get us a common cache? Or do we need to test it to find out?

Let's talk sync on this. btw, this is not related at all with this PR 😛 This is basically using bindings between nftstorage.link gateway and API (for perma-cache) as part of #101

@@ -222,12 +212,29 @@ function validateCacheControlHeader(cacheControl) {
* @param {Request} request
*/
function getHeaders(request) {
const existingProxies = request.headers.get('X-Forwarded-For')
? `, ${request.headers.get('X-Forwarded-For')}`
const headers = cloneHeaders(request.headers)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this is needed for the range headers support PR comming later.

Copy link
Contributor

@olizilla olizilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vasco-santos vasco-santos merged commit 80de324 into main Jun 30, 2022
@vasco-santos vasco-santos deleted the fix/use-worker-bindings branch June 30, 2022 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants