Skip to content

Added support for local distributed tables in join queries#3577

Merged
glookka merged 4 commits into
masterfrom
distr_join
Jul 16, 2025
Merged

Added support for local distributed tables in join queries#3577
glookka merged 4 commits into
masterfrom
distr_join

Conversation

@glookka

@glookka glookka commented Jul 14, 2025

Copy link
Copy Markdown
Contributor

Added support for local distributed tables in join queries

@glookka glookka requested a review from Copilot July 14, 2025 13:55
@glookka glookka marked this pull request as draft July 14, 2025 13:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Added support for performing joins over local partitions of distributed tables by propagating the original “parent” table name through queue creation, join sorter, and search handler logic.

  • Extended queue creator and multi-queue interfaces with a szParent parameter to carry the parent index name.
  • Updated join sorter and filter-checking logic to use the passed parent name for distributed-local joins.
  • Modified search handler to build and track JoinedIndexes_t with parent metadata.
  • Added an SQL test in test/test_278/test.xml for inner joins over a local distributed table.

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 2 comments.

File Description
test/test_278/test.xml Added inner-join test case for a local-distributed table d.
src/queuecreator.h/.cpp Added const char* szParent to queue-creation APIs and constructors.
src/joinsorter.h/.cpp Propagated parent table names into join sorter constructors and filters.
src/daemon/search_handler.h/.cpp Introduced JoinedIndexes_t, updated joined index population, and search flow.
Comments suppressed due to low confidence (3)

src/queuecreator.h:53

  • Adding the szParent parameter changes the public API signature without a default argument or overload, breaking existing callers. Please add a default value (= nullptr) or provide an overload to maintain backward compatibility.
void			sphCreateMultiQueue ( const SphQueueSettings_t & tQueue, const VecTraits_T<CSphQuery> & dQueries, VecTraits_T<ISphMatchSorter *> & dSorters, VecTraits_T<CSphString> & dErrors, SphQueueRes_t & tRes, StrVec_t * pExtra, QueryProfile_c * pProfile, const char * szParent );

test/test_278/test.xml:665

  • [nitpick] The new test covers only an inner join on the distributed-local table. Consider adding tests for left joins and failure scenarios (e.g., joining on a distributed table with remote agents) to ensure full coverage of the new feature.
<!-- join over distributed left table -->

src/joinsorter.cpp:722

  • [nitpick] GetJoinedIndexName() allocates a new CSphString on every call, which may be expensive in hot paths. Consider returning a const CSphString& or caching the result to reduce heap allocations.
	CSphString	GetJoinedIndexName() const	{ return m_sRightIndexSchemaName.IsEmpty() ?  m_pJoinedIndex->GetName() : m_sRightIndexSchemaName; }

struct JoinedIndexes_t
{
CSphVector<const CSphIndex *> m_dIndexes;
const char * m_szParent = nullptr;

Copilot AI Jul 14, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Storing raw const char* pointers to external CSphString data can lead to dangling pointers when the original string is modified or destroyed. Consider using a CSphString member for m_szParent to own the string and ensure safety.

Suggested change
const char * m_szParent = nullptr;
CSphString m_sParent;

Copilot uses AI. Check for mistakes.
{
if ( pServedDistIndex->m_dAgents.GetLength() )
{
m_sError << "join not allowed on distributed table containing remote agents '" << tQuery.m_sJoinIdx << "'";

Copilot AI Jul 14, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message literal opens with a single quote but does not clearly close it, leading to mismatched quotes. Update the literal or add a trailing apostrophe in the string to ensure the output is consistent.

Copilot uses AI. Check for mistakes.
@github-actions

github-actions Bot commented Jul 15, 2025

Copy link
Copy Markdown
Contributor

Windows test results

    3 files      3 suites   24m 46s ⏱️
1 083 tests 1 031 ✅ 52 💤 0 ❌
1 087 runs  1 035 ✅ 52 💤 0 ❌

Results for commit ab59480.

♻️ This comment has been updated with latest results.

@glookka glookka marked this pull request as ready for review July 16, 2025 17:28
@glookka glookka merged commit c420823 into master Jul 16, 2025
53 checks passed
@glookka glookka deleted the distr_join branch July 16, 2025 17:28
@glookka glookka mentioned this pull request Jul 16, 2025
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants