Find a more graceful way to update blocks/challenges in the database

[Bouncey]

Currently, the flow for updating the curriculum in the database is to delete them all and re-seed.

We should really upsert them using the id they have been assigned.

Requirements of the script

1. It should not remove all documents from the block and challenge collection in order to update the database
2. It should upsert any changes to the documents in the block and challenge collection.
3. It should remove documents from the collection that do not have a corresponding entry in the @freecodecamp/curriculum module
4. It should not cause a bottleneck in the database, this script will be run in production on a live database
5. There may be a requirement to implement an api endpoint to force loopback to reload the collections. Context: Loopback caches documents from the db in memory, any updates to those documents outside of the loopback api will require a manual reload. The script should also force the cachedMap to be regenerated.

[raisedadead]

Bump. Still a critpath right?

[Bouncey]

Yes-ish.

I think we do not need challenges/blocks in the db any longer. There is no part of the application life cycle that updates the blocks or challenges. We can create any data structures we need at build time.

I would like to remove curriculum data from the db.

[raisedadead]

We need them to lookup the challenge paths for a user if I am not mistaken in the UI? Is that being resolved in different way?

[Bouncey]

redirectToLearn uses the pathMigrationMap which is created at build time.

The challengeUrlResolver takes a challenge id (user.currentChallengeId), does a findById look up then builds the url from the document returned. This could be a build-time thing.

getFirstChallenge does a where query to find the first challenge. This could be a build-time thing.

boot/certificate uses a Challenge.findById to pull the project id's for each cert. This could be a build-time thing.

We never create, update or delete a Challenge/Block. Everything we query for could be a build time operation using the challenge markdown parser.

There are some legacy map-utils that I would look closely at and either remove them or do something at build time.