Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map GDS call is passing a FactorizedTable instead of a schema #4203

Open
semihsalihoglu-uw opened this issue Sep 6, 2024 · 0 comments
Open
Assignees
Labels
good-warm-up Good warm up feature refactoring

Comments

@semihsalihoglu-uw
Copy link
Contributor

I was in the following situation. I'm parallelizing the writes of the GDS outputs. In this case, each thread needs to create a local copy of an FTable. The current PlanMapper::mapGDSCall passes an actual FTable instead of TableSchema down to the operators (putting this in the GDSCallSharedState). This left me in a situation where I wanted to copy or clone the FTable but there is no such function.

Having talked to @acquamarin apparently normally the frontend passes just the schema and each operator that wants to do parallel writes, e.g., ResultCollector, use this schema to create its local copy with a line like this (see ResultCollector::initLocalStateInternal for an example):

    localTable = std::make_unique<FactorizedTable>(context->clientContext->getMemoryManager(),
        info.tableSchema.copy());

So maybe we should follow this style also in GDSCall. Or we can add a clone function to FTable and pass in FTable in other places too. Regardless, let's be consistent everywhere for code readability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good-warm-up Good warm up feature refactoring
Projects
None yet
Development

No branches or pull requests

2 participants