-
Notifications
You must be signed in to change notification settings - Fork 533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fail to update table in 2 different stream topic #531
Comments
Update: I think it failed because the re-partition topics for Table work differently on 2 topics. My question is: How to generate the init data for table for |
Hi, Is your need to first consume the topic |
Hi @StephenSorriaux , I mean Currently, the solution about create another Thanks. |
It would be nice if we had a command to feed initial data to a table. I'm not sure this is the best way of going about doing this, if we had a command for this You also don't have to populate the table with initial values, instead you can #: Default count for groups where we have no default in COUNT_DEFAULTS.
COUNT_DEFAULT: int = 300
#: Default count by group name.
COUNT_DEFAULTS: Mapping[str, int] = {
'foo': 600,
}
def get_count_or_default(group: str) -> int:
return count_table.setdefault(group, COUNT_DEFAULTS.get(group, COUNT_DEFAULT))
@app.agent()
async def s(stream):
async for line in stream:
count_table[line.group_name] = get_count_or_default(line.group_name) + 1 If you have the same default for all groups then even easier: count_table = app.Table('count_table', default=lambda: 300, partitions=partition_no) |
Here's another way to set initial values..., send to the changelog topic: INITIAL_COUNT = {
'foo': 300,
'bar': 600,
}
@app.command()
async def init():
async with app.producer:
await count_table.changelog_topic.maybe_declare()
await asyncio.sleep(1)
for key, value in INITIAL_COUNT.items():
await (await(count_table.changelog_topic.send(
key=key, value=value))) |
Checklist
master
branch of Faust.Steps to reproduce
Given one Faust Table, two topic Agents:
First feed data to agent of
topic_1
, give thecount_table
some init value (run from first start):Then add some data (run in real time) for
count_table
for agent oftopic_2
:Expected behavior
The
count_table[GROUP]
fromprocess_topic_2
start fromline['init_value']
value and count upActual behavior
The
count_table[GROUP]
fromprocess_topic_2
start from 0 and count upFull traceback
No error observed
Versions
The text was updated successfully, but these errors were encountered: