-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor platforms #34
Conversation
✅ Deploy Preview for metaforecast ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
(Forgot to push my last commit here which I described in the text above. Fixed.) |
Nice, looks good |
Mmh, actually, there are some merge conflicts, could you look into those? @berekuk. Probably a result of my refactoring polymarket's fetcher to take less memory yesterday. |
Oh right, I haven't merged your code yet. Just wanted to finish with #36 and other umbrella tasks, so pushed a bit early. Will fix soon. |
Merged. I've also discovered that I broke down
|
64ce4bd
to
0c9e577
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good, though I left a few comments.
Oh, I broke |
But it's probably not the reason xrisk is missing from prod. Not sure what's up with that, updating manually, will comment on #39 soon. |
Fixed: I added xrisk and givewellopenphil platform descriptions. Their fetchers are now responsible for reading |
Also: turns out I somewhat broke algolia updates in one of my previous PRs. Here's how it happened:
I fixed this on this branch in 49d8140, and I'm not sure if this affected prod. Probably not, since in the scheduler job there are a few more steps after Also, I moved algolia settings (app id and search key) to .env ( |
@@ -38,7 +38,7 @@ export const platforms: Platform[] = [ | |||
fantasyscotus, | |||
foretold, | |||
goodjudgment, | |||
goodjudgmentopen, | |||
goodjudmentopen, // note the typo! current table name is without `g`, `goodjudmentopen` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should really be fixed.
I did a few things here.
The key change is that all files in
backend/platforms/
now conform to thePlatform
interface, which looks like this:Note that
fetcher
doesn't store anything in the DB, it just returns the forecasts. And thenprocessPlatform(platform)
frombackend/platforms/index.ts
upserts it.This way we can:
Platform
interface so that we can evolve it in the future, e.g. add "longName" field, incremental updates, etc.I also:
all-platforms.ts
toindex.ts
; since it's the "main entry point to platforms", it makes sense to keep it in indexplatforms/all/platformFetchers.ts
andplatforms/all/platformNames.ts
, they are not needed now-fetch
suffix from platform filesForecast
type (seeplatforms/index.ts
) which is quite important for future type safetyThis is the first step to my umbrella scheduler/platforms layer refactoring plan which I'll describe in a separate issue, and which adds more context about why I'm moving in this direction.
@NunoSempere, please let me know if you dislike any of this! One thing that this PR breaks is your pattern of "call a platform code by uncommenting a
// polymarket();
line. But the same can be done with CLI, I think, so nothing significant is lost.There are also a few minor details we could discuss, e.g. I know I could avoid an explicit
name
field in a Platform object and detect it withfun.name
as it was done before; or at least useexport default
to avoid mentioning the platform name. There are some downsides to other approaches, though, some of them are too fragile to typos, and some are not compatible with typescript (e.g., it's impossible to specify a type when doingexport default
without creating an extra variable; also,export default
is subtly worse than non-default exports, I believe).