Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI documentation state change events #235

Merged
merged 8 commits into from
May 26, 2024
Merged

CI documentation state change events #235

merged 8 commits into from
May 26, 2024

Conversation

tayloraswift
Copy link
Owner

@tayloraswift tayloraswift commented May 23, 2024

goal: expose long poll API for getting notifications when documentation for a particular Edition is successfully uplinked

@tayloraswift tayloraswift marked this pull request as draft May 23, 2024 00:26
@tayloraswift
Copy link
Owner Author

alongside GraphStateLoop, which runs forever, we also need two additional immortal tasks, one for the server (which is currently the root task) and one for opening and iterating the MongoDB change stream cursor.

(sessions:Mongo.SessionPool) in 

GraphStateLoop.run
{
    (gsl:GraphStateLoop) in 

    async 
    let _:Void = forwardCursorItems(from: sessions, to: gsl, ...)
    //  Run forever...
    await ServerLoop.init(gsl, sessions, ...)
}

since we probably also want some sort of BuilderLoop following the same pattern, that’s going to involve nesting closures in closures, which might not scale. reminds me of withUnsafePointer pyramids…

@tayloraswift tayloraswift self-assigned this May 23, 2024
@tayloraswift
Copy link
Owner Author

cc @Joannis

@tayloraswift
Copy link
Owner Author

i believe we eventually want to store the build outcome, even for failed builds on a per-version basis, to allow receiving notification of build failure as well.

since build outcome is accompanied by build logs, this means we should

  • store log metadata in Snapshot instead of BuildMetadata

  • add a task that regularly deletes logs older than 30d from the database and s3

  • change s3 path computation to shard with a date prefix

@tayloraswift tayloraswift marked this pull request as ready for review May 26, 2024 21:06
@tayloraswift tayloraswift merged commit 73a3679 into master May 26, 2024
2 checks passed
@tayloraswift tayloraswift deleted the ci-events-api branch May 26, 2024 21:07
@tayloraswift
Copy link
Owner Author

tayloraswift commented May 26, 2024

continuation in #237

@tayloraswift
Copy link
Owner Author

alternative, somewhat orthogonal feature implemented here: #238

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant