I tried doing an Albers equal area projection, but the edges appeared jagged

[heian911]

https://proj.org/en/9.4/operations/projections/aea.html

[ahocevar]

There are two facts contributing to what you're seeing:

1. Tile sources using EPSG:3857 don't have data for the poles, because their y coordinates would be +/- Infinity. So you won't get data beyond approximately +/- 85°.
2. The further you get outside the validity extent of a projection, the more inaccurate the reprojection will get.

You'll get better results with vector data, but problem 2 will persist. Note that the validity extent of EPSG:3005 is much smaller. See the outline of the validity extent in blue:

You can play with the codesandbox that I created the above screenshot from here: https://codesandbox.io/p/sandbox/simple-forked-z8j7yn?file=%2Fmain.js%3A61%2C12. Caution: for the calculation of the blue extent polygon, it makes use of functions that are not part of the supported API.