Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support aggregates with stype = internal #3916

Open
ChenHuajun opened this issue Jun 16, 2020 · 4 comments
Open

Support aggregates with stype = internal #3916

ChenHuajun opened this issue Jun 16, 2020 · 4 comments

Comments

@ChenHuajun
Copy link

Now, aggregation functions that support parallel execution in Citus are configured through a hard-coded whitelist in multi_logical_optimizer.h.

We need to use the aggregate function in pg_roaringbitmap

https://github.com/ChenHuajun/pg_roaringbitmap
-rb_and_agg
-rb_or_agg

Could this simple whitelist of aggregation functions satisfying the associativity be made configurable?

@metdos
Copy link
Contributor

metdos commented Jun 16, 2020

@onurctirtir
Copy link
Member

onurctirtir commented Jun 16, 2020

Citus actually supports pushing execution to the worker nodes for custom aggregates but it does not support it for aggregates having their state_data_type to be internal

@onurctirtir onurctirtir changed the title Can aggregate function whitelist support configurable? Support aggregates with stype = internal Jun 16, 2020
@ChenHuajun
Copy link
Author

@ChenHuajun I suppose you don't add distribution column to your queries?
https://docs.citusdata.com/en/v9.3/develop/reference_sql.html?highlight=coordinator_aggregation_strategy#aggregate-functions

Yes. As @onurctirtir said what i need is support of aggregates with stype = internal

@marcocitus
Copy link
Member

More context in #120

We should add a metadata table that contains "safe" aggregates (and check it in AggregateEnabledCustom) and then implement serialization in worker_partial_agg_ffunc and deserialization in coord_combine_agg_sfunc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants