Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use the same default paths for clickhouse_keeper (symlink) as for clickhouse_keeper (executable) #52861

Merged
merged 3 commits into from Aug 5, 2023
Merged

Use the same default paths for clickhouse_keeper (symlink) as for clickhouse_keeper (executable) #52861

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
552331f
Use the same default paths for clickhouse_keeper (symlink) as for cli…
vitlibar Aug 1, 2023
89c99e4
Revert extracting the root keeper path from the 'path' key in the con…
vitlibar Aug 4, 2023
f55fe91
Throw an exception if wrong default path is going to be used.
vitlibar Aug 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
19 changes: 17 additions & 2 deletions programs/keeper/Keeper.cpp
View file
Open in desktop
Expand Up @@ -288,13 +288,27 @@ try
std::string path;

if (config().has("keeper_server.storage_path"))
{
path = config().getString("keeper_server.storage_path");
}
else if (std::filesystem::is_directory(std::filesystem::path{config().getString("path", DBMS_DEFAULT_PATH)} / "coordination"))
{
throw Exception(ErrorCodes::NO_ELEMENTS_IN_CONFIG,
"By default 'keeper.storage_path' could be assigned to {}, but the directory {} already exists. Please specify 'keeper.storage_path' in the keeper configuration explicitly",
KEEPER_DEFAULT_PATH, String{std::filesystem::path{config().getString("path", DBMS_DEFAULT_PATH)} / "coordination"});
}
else if (config().has("keeper_server.log_storage_path"))
{
path = std::filesystem::path(config().getString("keeper_server.log_storage_path")).parent_path();
Comment on lines 290 to 302
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order of checks is weird. It will throw the exception if we have keeper_server.log_storage_path and keeper_server.snapshot_storage_path set, but don't have keeper_server.storage_path. But it doesn't make sense to require keeper_server.storage_path in this case, there's no ambiguity

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pufit, would you like to make a follow-up PR with a fix? (Vitaly is on vacation right now)

}
else if (config().has("keeper_server.snapshot_storage_path"))
{
path = std::filesystem::path(config().getString("keeper_server.snapshot_storage_path")).parent_path();
}
else
path = std::filesystem::path{KEEPER_DEFAULT_PATH};
{
path = KEEPER_DEFAULT_PATH;
}

std::filesystem::create_directories(path);

Expand Down Expand Up @@ -330,6 +344,7 @@ try
auto global_context = Context::createGlobal(shared_context.get());

global_context->makeGlobalContext();
global_context->setApplicationType(Context::ApplicationType::KEEPER);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main fix is here because Context::initializeKeeperDispatcher() checks the application type and then passes is_standalone_app = (application_type == ApplicationType::KEEPER) to KeeperContext which uses it to calculate the default paths.

global_context->setPath(path);
global_context->setRemoteHostFilter(config());

Expand Down Expand Up @@ -365,7 +380,7 @@ try
}

/// Initialize keeper RAFT. Do nothing if no keeper_server in config.
global_context->initializeKeeperDispatcher(/* start_async = */ true);
global_context->initializeKeeperDispatcher(/* start_async = */ false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Keeper class can be used either by clickhouse-keeper executable or by clickhouse-keeper symlink. It's not used by clickhouse-server, so asynchronous starting is not required here. See also the assert in Context::initializeKeeperDispatcher().

FourLetterCommandFactory::registerCommands(*global_context->getKeeperDispatcher());

auto config_getter = [&] () -> const Poco::Util::AbstractConfiguration &
Expand Down
8 changes: 8 additions & 0 deletions src/Coordination/Standalone/Context.h
View file
Open in desktop
Expand Up @@ -98,6 +98,14 @@ class Context : public std::enable_shared_from_this<Context>
std::shared_ptr<FilesystemCacheLog> getFilesystemCacheLog() const;
std::shared_ptr<FilesystemReadPrefetchesLog> getFilesystemReadPrefetchesLog() const;

enum class ApplicationType
{
KEEPER
};

void setApplicationType(ApplicationType) {}
ApplicationType getApplicationType() const { return ApplicationType::KEEPER; }

IAsynchronousReader & getThreadPoolReader(FilesystemReaderType type) const;
std::shared_ptr<AsyncReadCounters> getAsyncReadCounters() const;
ThreadPool & getThreadPoolWriter() const;
Expand Down