Skip to content

[Submission] Durable Research Fleet / Serverless Workers #126

Description

@LondheShubham153

Project link

https://github.com/temporal-community/durable-research-fleet

Language

Python

Short description (max 256 chars)

A deep-research agent that runs on Temporal Serverless Workers on Google Cloud Run.

Ask a question from your phone. It splits into ~6 independent sub-questions, each researched in parallel by a Cloud Run instance that did not exist a second ago, then merged into one cited report. The pool scales 0 → N → 0 on its own.

Long Description

I built a research agent that runs entirely on infrastructure that doesn't exist until you ask it a question.

Here's what happens: you type a question into your phone. A Temporal workflow takes it and breaks it into about six sub-questions that can be researched independently. Each one gets picked up by a Cloud Run instance, and a second before that instance was working, it wasn't running at all. The whole pool was sitting at zero.

Six instances spin up, each works its own piece of the question, and once they're done the results get merged into a single report with citations, sent back to your phone. Then the pool scales back down to nothing. You're paying for six instances for roughly the length of time they were actually doing work, and that's it.

This is running on Temporal's Serverless Workers, which just went into pre-release, deployed on GCP Cloud Run. I wrote up the first version of this a bit more informally on LinkedIn (link below) and got enough questions about how it actually works that I figured it was worth a proper walkthrough.

A few things worth calling out if you're evaluating this for real work: if one of those six research steps fails partway through, Temporal doesn't throw away the other five or start the whole thing over. It picks up the failed step where it left off. And there's no autoscaling config to write here, no dashboard of thresholds to tune. Cloud Run and Temporal's own controller handle the scaling decision between them.

Full walkthrough (5 min) is linked below, and the original post that started this is here: [LinkedIn post link]

Author(s)

Shubham Londhe / Temporal / https://www.linkedin.com/in/shubhamlondhe1996/?lipi=urn%3Ali%3Apage%3Ad_flagship3_profile_view_base%3BwACGnapbSpaH3pfgt7oXIw%3D%3D

Metadata

Metadata

Assignees

Labels

code exchange submissionCode and/or content about Temporal!triageIssues that Temporal folk need to look atziggy reviewedPre-screened by ZiggyBot

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions