Skip to content
This repository has been archived by the owner on Jan 7, 2022. It is now read-only.

Figure out how to build / load / combine plugins #37

Closed
Zepheus opened this issue Oct 5, 2018 · 3 comments
Closed

Figure out how to build / load / combine plugins #37

Zepheus opened this issue Oct 5, 2018 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@Zepheus
Copy link
Contributor

Zepheus commented Oct 5, 2018

Right now, there is a PluginPack that registers a single binary (.so) for different purposes.
We need to figure out:

  • How to separate each part into a separate binary. E.g. I'd like to use Prometheus for logging and ElasticSearch for tracing. It should allow combinations flexibly.
  • How to build plugins in open source. Should we have a sub-directory? Separate cmake files for each? Separate repositories?
@sh00s
Copy link
Contributor

sh00s commented Oct 6, 2018

For the first part, I propose to transform the PluginPack into something with the following interface:

// All plugins will have to derive from this class. The implementation has to
// ensure that calls to all methods are thread-safe
class Plugin {
 public:
  enum class Type {
    PLUGIN_COLLECTION = 0,
    CONFIG_SOURCE = 1,
    STATS_PUBLISHER = 2,
    TRACE_LOGGER = 3,
    ...
  };
  // Specifies the type this plugin belongs to.
  virtual Type type() const = 0;

  // Returns the identifier for this plugin. This can be used to:
  // 1) load multiple plugins of the same type and pick which one to use
  //    via configuration / settings, and
  // 2) have per-plugin configuration.
  // Identifiers should be unique for all plugins loaded.
  virtual std::string identifier() const = 0;

  // A string that we can log somewhere when using this plugin.
  virtual std::string displayName() const = 0;

  // Versioning information
  virtual std::string version() const = 0;

  // If per-plugin configuration is present in the config, it would be fed to the
  // plugin immediately after initialization and on every config update.
  void onConfigUpdated(folly::dynamic plugin_configuration) = 0;
};

// A helper that would be returned by a `load_plugin()` function to ease loading several
// plugins at once from a single library
class PluginCollection : public Plugin {
 public:
  Type type() override {
    return Type::PLUGIN_COLLECTION;
  }
  virtual std::vector<std::unique_ptr<Plugin>> getPlugins() = 0;
};

Then we'd have classes that derive from the Plugin class to implement each of the plugin types - e.g. TraceLoggerPlugin, ConfigSourcePlugin, etc. Each implementation would then have to derive from these.

This is off the top of my head, so suggestions for improvement are welcome.

@sh00s sh00s self-assigned this Oct 11, 2018
@sh00s
Copy link
Contributor

sh00s commented Nov 12, 2018

Separated #56 and #57 out of this

@sh00s
Copy link
Contributor

sh00s commented Nov 13, 2018

This is mostly done. I added documentation in 84fbb0a to explain how to write new plugins, and filed the follow-ups mentioned above for the remaining parts.

@sh00s sh00s closed this as completed Nov 13, 2018
facebook-github-bot pushed a commit that referenced this issue Jul 24, 2019
Summary:
This is just facebookarchive/bistro#37, backported to `fbcode_builder`.

snarkmaster
Pull Request resolved: facebookarchive/bistro#37

Test Plan: Push and watch Travis

Reviewed By: simpkins

Differential Revision: D16453080

Pulled By: snarkmaster

fbshipit-source-id: a15eaead931f046c41e50f8e3b412ef68b172d65
facebook-github-bot pushed a commit that referenced this issue Oct 23, 2019
Summary:
I feel bad for such a hacky fix, but (a) I don't have better simple ideas, (b) it's only used in tests.

Stack trace:
  (gdb) bt
  #0  raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:50
  #1  0x000000000096052a in facebook::logdevice::handle_fatal_signal (sig=<optimized out>) at logdevice/server/fatalsignal.cpp:58
  #2  <signal handler called>
  #3  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
  #4  0x00007f0b92e3b935 in __GI_abort () at abort.c:90
  #5  0x00007f0b940fc395 in __gnu_cxx::__verbose_terminate_handler () at ../../.././libstdc++-v3/libsupc++/vterminate.cc:95
  #6  0x00007f0b940fa016 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:47
  #7  0x00007f0b940fa061 in std::terminate () at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:57
  #8  0x00007f0b940fa33b in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x26e5800 <typeinfo for std::bad_function_call>, dest=0x415cf0 <std::bad_function_call::~bad_function_call()plt>) at ../../.././libstdc++-v3/libsupc++/eh_throw.cc:93
  #9  0x00000000004b5f92 in folly::throw_exception<std::bad_function_call> (ex=...) at folly/lang/Exception.h:36
  #10 0x00000000004b5f57 in folly::detail::throw_exception_<std::bad_function_call> () at folly/lang/Exception.h:64
  #11 0x0000000000d70019 in folly::throw_exception<std::bad_function_call> () at folly/lang/Exception.h:82
  #12 folly::detail::function::FunctionTraits<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::uninitCall(folly::detail::function::Data&, facebook::logdevice::E&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&) () at folly/Function.h:372
  #13 0x0000000000f023f8 in folly::detail::function::FunctionTraits<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>::operator()(facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (this=<optimized out>, args=..., args=...) at folly/Function.h:377
  #14 facebook::logdevice::FileBasedVersionedConfigStore::getConfig(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, folly::Function<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>, folly::Optional<facebook::logdevice::vcs_config_version_t>) const (
      this=<optimized out>, key=..., cb=..., base_version=...) at logdevice/common/FileBasedVersionedConfigStore.cpp:258
  #15 0x0000000000d76019 in facebook::logdevice::configuration::nodes::VersionedNodesConfigurationStore<facebook::logdevice::FileBasedVersionedConfigStore>::getConfig(folly::Function<void (facebook::logdevice::E, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)>, folly::Optional<facebook::logdevice::vcs_config_version_t>) const (
      this=<optimized out>, cb=..., base_version=...) at logdevice/common/configuration/nodes/NodesConfigurationStore.h:88
  #16 0x0000000000d6de2f in facebook::logdevice::configuration::nodes::ncm::Dependencies::readFromStore (this=0x7f0b91967fa0, should_do_consistent_config_fetch=false) at logdevice/common/configuration/nodes/NodesConfigurationManagerDependencies.cpp:342
  #17 0x0000000000d5e64e in facebook::logdevice::configuration::nodes::NodesConfigurationManager::onHeartBeat (this=0x7f0b90d0df10) at logdevice/common/configuration/nodes/NodesConfigurationManager.cpp:538
  #18 0x0000000000d73073 in facebook::logdevice::configuration::nodes::ncm::Dependencies::scheduleHeartBeat()::$_4::operator()() const (this=<optimized out>) at logdevice/common/configuration/nodes/NodesConfigurationManagerDependencies.cpp:365
  #19 std::_Function_handler<void (), facebook::logdevice::configuration::nodes::ncm::Dependencies::scheduleHeartBeat()::$_4>::_M_invoke(std::_Any_data const&) (__functor=...) at third-party-buck/platform007/build/libgcc/include/c++/trunk/bits/std_function.h:316
  #20 0x0000000000e250a6 in std::function<void ()>::operator()() const (this=<optimized out>) at third-party-buck/platform007/build/libgcc/include/c++/trunk/bits/std_function.h:706
  #21 facebook::logdevice::(anonymous namespace)::WheelTimerDispatchImpl::makeWheelTimerInternalExecutor(facebook::logdevice::Worker*)::$_0::operator()()::{lambda()#1}::operator()() const (this=0x7f0b8bf34070) at logdevice/common/Timer.cpp:123
  #22 folly::detail::function::FunctionTraits<void ()>::callSmall<facebook::logdevice::(anonymous namespace)::WheelTimerDispatchImpl::makeWheelTimerInternalExecutor(facebook::logdevice::Worker*)::$_0::operator()()::{lambda()#1}>(folly::detail::function::Data&) (p=...) at folly/Function.h:361
  #23 0x0000000000e63602 in folly::detail::function::FunctionTraits<void ()>::operator()() (this=<optimized out>) at folly/Function.h:377
  #24 facebook::logdevice::Worker::addWithPriority(folly::Function<void ()>, signed char)::$_9::operator()() (this=0x7f0b8bf34060) at logdevice/common/Worker.cpp:1373
  #25 folly::detail::function::FunctionTraits<void ()>::callBig<facebook::logdevice::Worker::addWithPriority(folly::Function<void ()>, signed char)::$_9>(folly::detail::function::Data&) (p=...) at folly/Function.h:368
  #26 0x00000000010d7e75 in folly::detail::function::FunctionTraits<void ()>::operator()() (this=0x7f0b8bf34060) at folly/Function.h:377
  #27 facebook::logdevice::EventLoopTaskQueue::executeTasks (this=0x7f0b74023000, tokens=<optimized out>) at logdevice/common/EventLoopTaskQueue.cpp:154
  #28 0x00000000010d7699 in facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler()::$_1::operator()(unsigned int) const (this=<optimized out>, n=1) at logdevice/common/EventLoopTaskQueue.cpp:101
  #29 facebook::logdevice::LifoEventSemImpl<std::atomic>::AsyncWaiter::processBatch<facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler()::$_1&>(facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler()::$_1&, unsigned int) (this=<optimized out>, func=..., maxBatchSize=<optimized out>) at logdevice/common/LifoEventSem.h:351
  #30 facebook::logdevice::EventLoopTaskQueue::haveTasksEventHandler (this=0x7f0b74023000) at logdevice/common/EventLoopTaskQueue.cpp:106
  #31 0x0000000001a907f2 in event_process_active_single_queue (base=<optimized out>, activeq=0x7f0b74020010, max_to_process=2147483647, endtime=0x0) at logdevice/external/libevent-2.1.3-alpha/event.c:1449
  #32 0x0000000001a8efae in event_process_active (base=<optimized out>) at logdevice/external/libevent-2.1.3-alpha/event.c:1599
  #33 ld_event_base_loop (base=0x7f0b7401b000, flags=<optimized out>) at logdevice/external/libevent-2.1.3-alpha/event.c:1819
  #34 0x000000000176b5f1 in facebook::logdevice::EvBaseLegacy::loop (this=0x7f0b74000018) at logdevice/common/libevent/EvBaseLegacy.cpp:58
  #35 0x00000000010d5a79 in facebook::logdevice::EventLoop::run (this=<optimized out>) at logdevice/common/EventLoop.cpp:140
  #36 0x00000000016fa223 in std::function<void ()>::operator()() const (this=0x7f0b90d0c520) at third-party-buck/platform007/build/libgcc/include/c++/trunk/bits/std_function.h:706
  #37 facebook::logdevice::thread_func (arg=0x7f0b90d0c520) at logdevice/common/PThread.cpp:9
  #38 0x00007f0b936136b6 in start_thread (arg=0x7f0b74dff700) at pthread_create.c:465
  #39 0x00007f0b92f2cebf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Reviewed By: gdrane

Differential Revision: D17975422

fbshipit-source-id: 499c6aceb577f859e9d4e71006b86f73537221ab
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants