Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

elasticsearch history api #1682 #1725

Merged
merged 16 commits into from Aug 14, 2019
Merged
Changes from 1 commit
Commits
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

add elasticsearch-mode command for save, query or all

  • Loading branch information...
oxarbitrage committed Jul 15, 2019
commit 1737d455e9c087159287133f55f4a879b9c0d913
@@ -59,6 +59,7 @@ class elasticsearch_plugin_impl
bool _elasticsearch_operation_object = false;
uint32_t _elasticsearch_start_es_after_block = 0;
bool _elasticsearch_operation_string = true;
std::string _elasticsearch_mode = "only_save";

This comment has been minimized.

Copy link
@pmconrad

pmconrad Jul 26, 2019

Contributor

For more efficient comparison, make this an enum type.

This comment has been minimized.

Copy link
@oxarbitrage

oxarbitrage Jul 31, 2019

Author Member

It is a bit tricky to do this because the boost::program_options will not accept the enum but basically just a string, number or boolean.
I picked the number option to do a static cast to the enum by checking the upper boundary to avoid inserting invalid options.

Implemented at de76301

This comment has been minimized.

Copy link
@oxarbitrage

oxarbitrage Jul 31, 2019

Author Member

changed the exception introduced here to graphene::chain::plugin_exception in the context of 5838a38

CURL *curl; // curl handler
vector <string> bulk_lines; // vector of op lines
vector<std::string> prepare;
@@ -436,17 +437,13 @@ void elasticsearch_plugin::plugin_set_program_options(
("elasticsearch-operation-object", boost::program_options::value<bool>(), "Save operation as object(false)")
("elasticsearch-start-es-after-block", boost::program_options::value<uint32_t>(), "Start doing ES job after block(0)")
("elasticsearch-operation-string", boost::program_options::value<bool>(), "Save operation as string. Needed to serve history api calls(true)")
("elasticsearch-mode", boost::program_options::value<std::string>(), "Mode of operation: only_save, only_query, all(only_save)")
This conversation was marked as resolved by oxarbitrage

This comment has been minimized.

Copy link
@pmconrad

pmconrad Jul 26, 2019

Contributor

Please wrap these long lines.

This comment has been minimized.

Copy link
@oxarbitrage
;
cfg.add(cli);
}

void elasticsearch_plugin::plugin_initialize(const boost::program_options::variables_map& options)
{
database().applied_block.connect( [&]( const signed_block& b) {
if (!my->update_account_histories(b))
FC_THROW_EXCEPTION(graphene::chain::plugin_exception, "Error populating ES database, we are going to keep trying.");
} );

my->_oho_index = database().add_index< primary_index< operation_history_index > >();
database().add_index< primary_index< account_transaction_history_index > >();

@@ -477,6 +474,19 @@ void elasticsearch_plugin::plugin_initialize(const boost::program_options::varia
if (options.count("elasticsearch-operation-string")) {
my->_elasticsearch_operation_string = options["elasticsearch-operation-string"].as<bool>();
}
if (options.count("elasticsearch-mode")) {
my->_elasticsearch_mode = options["elasticsearch-mode"].as<std::string>();
}

if(my->_elasticsearch_mode != "only_query") {
if (my->_elasticsearch_mode == "all")
my->_elasticsearch_operation_string = true;

This comment has been minimized.

Copy link
@pmconrad

pmconrad Jul 26, 2019

Contributor

Silently overriding this will cause confusion.
IMO add log output if user has configured this to false. Or perhaps fail to make this very explicit.

This comment has been minimized.

Copy link
@oxarbitrage

oxarbitrage Jul 31, 2019

Author Member

Agree, i am now not allowing the mode to be all if operation_string is false so no silent changes are made.

5838a38


database().applied_block.connect([&](const signed_block &b) {
This conversation was marked as resolved by oxarbitrage

This comment has been minimized.

Copy link
@pmconrad

pmconrad Jul 26, 2019

Contributor

Please capture only what's necessary.

This comment has been minimized.

Copy link
@oxarbitrage
if (!my->update_account_histories(b))
FC_THROW_EXCEPTION(graphene::chain::plugin_exception, "Error populating ES database, we are going to keep trying.");
});
}
}

void elasticsearch_plugin::plugin_startup()
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.