feat: migrate to cloudflare pages and vite (Work in progress) #81

BhanuNexus · 2024-06-20T11:58:17Z

Migrate to Remix Vite and Cloudflare Pages

Status: Work in progress
Issue: #55

benvinegar · 2024-06-20T13:55:08Z

Wow, awesome – I took a stab at this for a month+ ago and didn't end up finishing.

Let me know how I can help!

BhanuNexus · 2024-06-20T18:32:42Z

app/analytics/collect.test.ts

Trying to understand on how to update this tests

BhanuNexus · 2024-06-20T18:36:38Z

Hi @benvinegar, Whenever you get sometime, could you please checkout this branch and test? I don't have data for my cloudflare account, hence not able to test the dashboard completely.

Apart from collect.test.ts, remaining looks good in my tests.
FYI - I did npm package updates as well.

benvinegar · 2024-06-21T02:15:02Z

@BhanuNexus I think I addressed the tests.

But I don't know how to actually run this. If I try npm run dev it will load the homepage, but /collect 404s and trying to load the dashboard URL crashes the server.

It looks like this is converted to Cloudflare Pages (e.g. the functions dir), so I also tried wrangler pages dev but then I get the following compilation errors:

❯ node_modules/wrangler/bin/wrangler.js pages dev

 ⛅️ wrangler 3.61.0
-------------------

✘ [ERROR] 1 error(s) and 0 warning(s) when compiling Worker.



✘ [ERROR] Could not resolve "../build/server"

    [[page]].ts:3:23:
      3 │ import * as build from "../build/server";
        ╵                        ~~~~~~~~~~~~~~~~~


✘ [ERROR] Build failed with 1 error:

  [[page]].ts:3:23: ERROR: Could not resolve "../build/server"

Maybe I'm missing some steps/instructions?

BhanuNexus · 2024-06-21T03:38:13Z

@benvinegar could you please try below?

# added prepreview script now
npm run preview

BhanuNexus · 2024-06-21T03:38:40Z

I think I addressed the tests.

Thank you :)

favicon.ico to reduce localhost 404

benvinegar · 2024-06-21T22:46:30Z

@benvinegar could you please try below?

Yep, this gets the server running -- thank you -- but looks like it's producing a bad query because it can't extract the timezone correctly. Poking at this.

benvinegar · 2024-06-21T23:47:25Z

Ok, I think it works: the only remaining issue is figuring out how to inject VERSION into the context.

This doesn't seem to ever get a value:

context.cloudflare.env.CF_PAGES_COMMIT_SHA

Any ideas?

benvinegar · 2024-06-21T23:49:27Z

Here's your PR branch running locally, serving my production dataset:

counterscale-cf-pages.mov

benvinegar · 2024-06-22T00:00:29Z

Trying to deploy this and I basically just get 500 Internal Error with no visibility into what's going on / no actual HTML returned or anything.

BhanuNexus · 2024-06-22T19:02:20Z

Thank you very much for your changes, sorry for trouble with wrong parameter for timezone.

I am assuming there are no secrets set, can you try below?

# for production
npx wrangler pages secret put CF_ACCOUNT_ID --env production
npx wrangler pages secret put CF_BEARER_TOKEN --env production

# for preview
npx wrangler pages secret put CF_ACCOUNT_ID --env preview
npx wrangler pages secret put CF_BEARER_TOKEN --env preview

I'm able to run it here https://counterscaleapp.pages.dev

…r local deployment

BhanuNexus · 2024-06-22T19:11:10Z

This doesn't seem to ever get a value:

context.cloudflare.env.CF_PAGES_COMMIT_SHA

Any ideas?

This value is available with CF build environment (similar to github actions variables)
I can see value is available once deployed to pages, sample https://counterscaleapp.pages.dev/

benvinegar · 2024-06-25T15:05:12Z

I am assuming there are no secrets set, can you try below?

Sorry will try to poke at this tonight. I do wish there was better reporting coming out of pages or a way to capture this without blowing up.

benvinegar · 2024-06-26T03:13:22Z

Yep, that did it 🎉

# for preview
npx wrangler pages secret put CF_ACCOUNT_ID --env preview
npx wrangler pages secret put CF_BEARER_TOKEN --env preview

^Note I didn't need this I think because (I think) it reads .dev.vars which already had my local secrets.

Okay, so my thinking is: this is basically a major version. It's basically a re-architecture of what I'm used to, and I'm not super comfortable just making this main.

So here's my suggestion:

I actually tag a 1.0.0 release of Counterscale of HEAD on main
I start a v2 branch and merge this PR into that branch
Going forward, new feature development happens on the v2 branch
1. I'll change counterscale.dev to autodeploy from the v2 branch
After some time when I feel it's stable, there are no concerns, etc., I'll merge the v2 branch into main and tag a 2.0.0 release

@BhanuNexus Thoughts on this plan?

BhanuNexus · 2024-06-26T04:58:28Z

Note I didn't need this I think because (I think) it reads .dev.vars which already had my local secrets.

yeah, this is an issue and this also makes me think of using Cloudflare Pages settings instead of considering wrangler.toml.
If we remove pages_build_output_dir = "build/client" from wrangler.toml then variables can be set from cloudflare UI, that way it might be easy for people who fork this repo and use.

BhanuNexus · 2024-06-26T05:00:49Z

Yep, that did it 🎉
# for preview
npx wrangler pages secret put CF_ACCOUNT_ID --env preview
npx wrangler pages secret put CF_BEARER_TOKEN --env preview
^Note I didn't need this I think because (I think) it reads .dev.vars which already had my local secrets.

Okay, so my thinking is: this is basically a major version. It's basically a re-architecture of what I'm used to, and I'm not super comfortable just making this main.

So here's my suggestion:

I actually tag a 1.0.0 release of Counterscale of HEAD on main

I start a v2 branch and merge this PR into that branch

Going forward, new feature development happens on the v2 branch

I'll change counterscale.dev to autodeploy from the v2 branch

After some time when I feel it's stable, there are no concerns, etc., I'll merge the v2 branch into main and tag a 2.0.0 release

@BhanuNexus Thoughts on this plan?

Totally agreed.
Meanwhile, I can update documentation and need to check posibility of moving cloudflare function /collect to remix, this will help to reduce dependency on CF and devlopment/debugging will be easy in local.

benvinegar · 2024-07-11T17:40:10Z

Just following up – I've been on vacation, but back now and plan to make these branch changes soon.

Meanwhile, I can update documentation and need to check posibility of moving cloudflare function /collect to remix, this will help to reduce dependency on CF and devlopment/debugging will be easy in local.

👍

benvinegar · 2024-07-13T13:09:02Z

app/analytics/collect.ts

@@ -130,7 +130,7 @@ interface DataPoint {
 // More here: https://developers.cloudflare.com/analytics/analytics-engine/get-started/#limits

 export function writeDataPoint(
-    analyticsEngine: CFAnalyticsEngine,
+    analyticsEngine: AnalyticsEngineDataset,


I think generally speaking, this change isn't really related to migrating to vite, and more of a separate personal choice that's been inserted (same with renaming Environment to Env).

I'll keep it (it's more clear I suppose) but generally it's best to not to couple these changes.

Agreed, this AnalyticsEngineDataset type is directly from @cloudflare/workers-types
So, syntaxes and change tracking are aligned with CF.

Regarding Env, I personally prefer Environment whatever you have set but cloudflare auto generates types from wrangler.toml to worker-configuration.d.ts by using wrangler types, that is the reason I have removed Environment and reused Env from worker-configuration.d.ts

benvinegar · 2024-07-13T13:10:55Z

package.json

+        "dev": "remix vite:dev",
+        "build": "remix vite:build",
+        "prepreview": "remix vite:build",
+        "preview": "wrangler pages dev ./build/client",


When should I use npm run preview and when should I use npm run dev? Are the remix commands deprecated now?

wrangler pages dev ./build/client command simulate the CF environment locally like functions
Where as remix vite:build just works with remix, usually this command alone will have 404 for /collect endpoint of CF functions.

If we manage to migrate /collect endpoint to remix, then preview wouldn't be needed anymore. I believe this is doable with minor changes, I will try to get this done in couple of days.

BhanuNexus · 2024-07-13T20:39:40Z

app/routes/collect-beta.tsx

Hi @benvinegar, I have previously added this endpoint to test movement of /collect from CF function to remix.

I have tested this with mine https://cs.usevega.com and seems to be working fine, do you want to have this change included with this PR or may be in later iteration?

@BhanuNexus I started the v2 branch for this ongoing migration (here). I'd make a new PR targeting that.

I'm going to close this PR in the meantime.

benvinegar · 2024-07-14T14:21:36Z

Closing because this PR has become the v2 branch.

benvinegar · 2024-08-01T02:39:32Z

@BhanuNexus All your changes are now on main (version is v2.0.0-beta.2).

feat: migrate to cloudflare pages and vite (Work in progress)

b243d6a

BhanuNexus added 2 commits June 20, 2024 21:12

fix: collect endpoint

7d44539

feat: package updates and setup changes

a78bd5d

BhanuNexus commented Jun 20, 2024

View reviewed changes

app/analytics/collect.test.ts Outdated

Copy link

Contributor Author

BhanuNexus Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to understand on how to update this tests

Mock cf object to help tests pass

04fff67

benvinegar force-pushed the feature/vite branch from a5520c7 to 04fff67 Compare June 21, 2024 02:09

Fix isbot import (oops)

eca189d

BhanuNexus added 3 commits June 21, 2024 09:12

chore: add prepreview script to package.json

1189877

feat: Add collect-beta endpoint in remix

cca16c0

feat: added route layout and error boundary

035e493

favicon.ico to reduce localhost 404

benvinegar added 2 commits June 21, 2024 18:55

Access correct timezone property from cf object

c91f97a

Pass getLoaderContext into [[page]] function

7f9c6a2

Fix lint errors + lint build script

2a8e24b

feat: add node environment and deploy environment scripts, to help fo…

eb6dc38

…r local deployment

Merge branch 'benvinegar:main' into feature/vite

ae3968d

benvinegar approved these changes Jul 13, 2024

View reviewed changes

Merge branch 'benvinegar:main' into feature/vite

6de44b6

BhanuNexus commented Jul 13, 2024

View reviewed changes

benvinegar closed this Jul 14, 2024

benvinegar mentioned this pull request Aug 12, 2024

Migrate to Remix + Vite #55

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: migrate to cloudflare pages and vite (Work in progress) #81

feat: migrate to cloudflare pages and vite (Work in progress) #81

BhanuNexus commented Jun 20, 2024 •

edited

Loading

benvinegar commented Jun 20, 2024

BhanuNexus Jun 20, 2024

BhanuNexus commented Jun 20, 2024

benvinegar commented Jun 21, 2024

BhanuNexus commented Jun 21, 2024 •

edited

Loading

BhanuNexus commented Jun 21, 2024

benvinegar commented Jun 21, 2024

benvinegar commented Jun 21, 2024 •

edited

Loading

benvinegar commented Jun 21, 2024

benvinegar commented Jun 22, 2024

BhanuNexus commented Jun 22, 2024 •

edited

Loading

BhanuNexus commented Jun 22, 2024

benvinegar commented Jun 25, 2024

benvinegar commented Jun 26, 2024 •

edited

Loading

BhanuNexus commented Jun 26, 2024

BhanuNexus commented Jun 26, 2024 •

edited

Loading

benvinegar commented Jul 11, 2024

benvinegar Jul 13, 2024

BhanuNexus Jul 13, 2024 •

edited

Loading

benvinegar Jul 13, 2024

BhanuNexus Jul 13, 2024

BhanuNexus Jul 13, 2024

benvinegar Jul 14, 2024

benvinegar commented Jul 14, 2024

benvinegar commented Aug 1, 2024

feat: migrate to cloudflare pages and vite (Work in progress) #81

feat: migrate to cloudflare pages and vite (Work in progress) #81

Conversation

BhanuNexus commented Jun 20, 2024 • edited Loading

benvinegar commented Jun 20, 2024

BhanuNexus Jun 20, 2024

Choose a reason for hiding this comment

BhanuNexus commented Jun 20, 2024

benvinegar commented Jun 21, 2024

BhanuNexus commented Jun 21, 2024 • edited Loading

BhanuNexus commented Jun 21, 2024

benvinegar commented Jun 21, 2024

benvinegar commented Jun 21, 2024 • edited Loading

benvinegar commented Jun 21, 2024

benvinegar commented Jun 22, 2024

BhanuNexus commented Jun 22, 2024 • edited Loading

BhanuNexus commented Jun 22, 2024

benvinegar commented Jun 25, 2024

benvinegar commented Jun 26, 2024 • edited Loading

BhanuNexus commented Jun 26, 2024

BhanuNexus commented Jun 26, 2024 • edited Loading

benvinegar commented Jul 11, 2024

benvinegar Jul 13, 2024

Choose a reason for hiding this comment

BhanuNexus Jul 13, 2024 • edited Loading

Choose a reason for hiding this comment

benvinegar Jul 13, 2024

Choose a reason for hiding this comment

BhanuNexus Jul 13, 2024

Choose a reason for hiding this comment

BhanuNexus Jul 13, 2024

Choose a reason for hiding this comment

benvinegar Jul 14, 2024

Choose a reason for hiding this comment

benvinegar commented Jul 14, 2024

benvinegar commented Aug 1, 2024

BhanuNexus commented Jun 20, 2024 •

edited

Loading

BhanuNexus commented Jun 21, 2024 •

edited

Loading

benvinegar commented Jun 21, 2024 •

edited

Loading

BhanuNexus commented Jun 22, 2024 •

edited

Loading

benvinegar commented Jun 26, 2024 •

edited

Loading

BhanuNexus commented Jun 26, 2024 •

edited

Loading

BhanuNexus Jul 13, 2024 •

edited

Loading