Less copying and a correctness fix for dotProduct

[rschu1ze]

This is follow-up to #60202.

There were two problems:

• a performance problem (discovered by @gingerwizard) with mixed const / non-const arguments. It existed already before Vectorize dotProduct #60202. The reason is some unnecessary unpacking of the const argument, leading to unnecessary memory copies:
  https://pastila.nl/?000cd311/e87242ca16050916b887bc8f322e4c40#jAGOXIzrL+jNqlY0t8UIUQ==
• a correctness problem (discovered by me while making this PR) which was introduced with Vectorize dotProduct #60202 (sorry). To my defense, the tests we had before Vectorize dotProduct #60202 didn't trigger it. As a result, this PR must be backported to v24.2.
  Example: https://fiddle.clickhouse.com/ab8f2641-9381-4ca9-9292-dbeb4be5c644 (run with v24.1 and v24.2)

Reviewer note: This PR includes commits

• 2dc1721 Only refactorings/renamings for more consistency between related code
• c390974 Replaces the previous "shotgun cast" approach (which calls convertToFullColumn() / memcpy in the worst case 10 x 10 = 100 times) by a single call to convertToFullColumn()
• a69bcc2 More refactorings
• 245ea0c Implements a shortcut for const / non-const arguments which gets rid of the copy-ing for the const argument. Also fixes the correctness bug and extends the functional and performance tests.

Runtime with const/non-const arguments of array cardinality 150 on a table with 50 million rows (a typical scenario for vector search) goes down from 96.9 sec to 0.32 sec.

Changelog category (leave one):

• Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Optimized function dotProduct to omit unnecessary and expensive memory copies.

[robot-ch-test-poll1]

This is an automated comment for commit 2c0612c with description of existing statuses. It's updated for the latest CI running

⏳ Click here to open a full report in a separate page

Check name	Description	Status
CI running	A meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR	⏳ pending

Successful checks

Check name	Description	Status
Docs check	Builds and tests the documentation	✅ success
PR Check	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Style check	Runs a set of checks to keep the code style clean. If some of tests failed, see the related log from the report	✅ success

[yariks5s]

We already know here that col_x and col_y are not constant, is there any purpose to do convertToFullColumnIfConst?

[rschu1ze]

I think we can remove it. Interestingly, even if col_x and col_y are not ColumnConst (i.e. they are ColumnArray), calling ColumnArray::convertToFullColumnIfConst() does something:

ColumnPtr ColumnArray::convertToFullColumnIfConst() const
{
    /// It is possible to have an array with constant data and non-constant offsets.
    /// Example is the result of expression: replicate('hello', [1])
    return ColumnArray::create(data->convertToFullColumnIfConst(), offsets);
}

I checked the replicate and arrayWithConstant (the only two SQL functions which look like they could create an array with const data), but they they both create non-const data. It looks to me as if ColumnArray::convertToFullColumnIfConst() can be removed. I might try this in a separate PR.

[rschu1ze]

No performance tests on ARM ran and the logs don't make it really clear what went wrong. In my local testing on ARM (after merging from master and installing clickhouse-driver and scipy from pip), they ran successfully.

I'll merge from master once more.

[rschu1ze]

#61519 is missing, merging from master again.

[rschu1ze]

Risc-v build fails, a fix is on the way.

This and this performance tests complain

Run Errors
Test	Error
dotProduct	Traceback (most recent call last):
dotProduct	File "/workspace/right/scripts/perf.py", line 430, in

As far as I see in the logs, this is because a new query was added to tests/performance/dotProduct.xml, which runs too slow in the old version

Traceback (most recent call last):
│ File "/workspace/right/scripts/perf.py", line 430, in <module>
│   res = c.execute(
│ File "/usr/local/lib/python3.10/dist-packages/clickhouse_driver/client.py", line 267, in execute
│   rv = self.process_ordinary_query(
│ File "/usr/local/lib/python3.10/dist-packages/clickhouse_driver/client.py", line 453, in process_ordinary_query
│   return self.receive_result(with_column_types=with_column_types,
│ File "/usr/local/lib/python3.10/dist-packages/clickhouse_driver/client.py", line 117, in receive_result
│   return result.get_result()
│ File "/usr/local/lib/python3.10/dist-packages/clickhouse_driver/result.py", line 50, in get_result
│   for packet in self.packet_generator:
│ File "/usr/local/lib/python3.10/dist-packages/clickhouse_driver/client.py", line 133, in packet_generator
│   packet = self.receive_packet()
│ File "/usr/local/lib/python3.10/dist-packages/clickhouse_driver/client.py", line 150, in receive_packet
│   raise packet.exception
clickhouse_driver.errors.ServerException: Code: 159.
dotProduct.query3.run0: DB::Exception: Timeout exceeded: elapsed 15.296030259 seconds, maximum: 15. Stack trace:

but because of this PR, runtime improved to 0.23 sec which is below the maximum query runtime. Also, local performance test execution on x86 and ARM were good.

Let me merge from a master once more and merge as soon as the button is no longer greyed out.

[rschu1ze]

I was too slow and missed the short timeframe in which the Merge button was not greyed out. Another try, sorry.