Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Affinity for processors pipeline. Fix integration tests. #8362

Merged
merged 6 commits into from
Dec 26, 2019
Merged

Affinity for processors pipeline. Fix integration tests. #8362

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9482cf2
Enable processors by default.
KochetovNicolai Dec 23, 2019
a82079c
Set affinity for PipelineExecutor threads.
KochetovNicolai Dec 23, 2019
e53b68d
Fix apple build.
KochetovNicolai Dec 24, 2019
2148f09
Disable affinity for apple and FreeBSD.
KochetovNicolai Dec 24, 2019
5b2847f
Fix mysql_protocol integration tests.
KochetovNicolai Dec 25, 2019
348b13a
Disable processors by default.
KochetovNicolai Dec 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
15 changes: 15 additions & 0 deletions dbms/src/Processors/Executors/PipelineExecutor.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@
#include <Processors/ISource.h>
#include <Common/setThreadName.h>

#if !defined(__APPLE__) && !defined(__FreeBSD__)
#include <sched.h>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this also can be omitted for apple/freebsd

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.
Actually, it's better to add affinity for other os too.

#endif

namespace DB
{

Expand Down Expand Up @@ -452,6 +456,17 @@ void PipelineExecutor::execute(size_t num_threads)

void PipelineExecutor::executeSingleThread(size_t thread_num, size_t num_threads)
{
#if !defined(__APPLE__) && !defined(__FreeBSD__)
/// Specify CPU core for thread if can.
/// It may reduce the number of context swithches.
cpu_set_t cpu_set;
CPU_ZERO(&cpu_set);
CPU_SET(thread_num, &cpu_set);
if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set) == -1)
LOG_TRACE(log, "Cannot set affinity for thread " << num_threads);

#endif

UInt64 total_time_ns = 0;
UInt64 execution_time_ns = 0;
UInt64 processing_time_ns = 0;
Expand Down
3 changes: 3 additions & 0 deletions dbms/src/Processors/Formats/Impl/MySQLOutputFormat.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ void MySQLOutputFormat::initialize()

void MySQLOutputFormat::consume(Chunk chunk)
{

initialize();

for (size_t i = 0; i < chunk.getNumRows(); i++)
{
ProtocolText::ResultsetRow row_packet(data_types, chunk.getColumns(), i);
Expand Down