Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Events overhaul: loose coupling + plugin pipe timeout removal #1613

Merged
merged 14 commits into from
May 12, 2020

Conversation

scottinet
Copy link
Contributor

@scottinet scottinet commented May 1, 2020

Description

This PR changes how events are handled by Kuzzle's core component:

  • centralize the responsability of emitting events (hooks) and pipes in a new KuzzleEventEmitter object, localized in the new core/events directory
    • move the pipeRunner and waterfall objects into that new directory
    • move all pipes logic from the plugin's manager and the kuzzle main object into that new class
  • pipes can now be listened by core objects. They act like plugin pipes, with the following differences:
    • core functions are invoked after all plugin pipes are resolved
    • if multiple core functions are registered on the same pipe events, they are called in parallel (plugin pipes are waterfalled)
    • core functions are expected to return promises
    • the pipe emitter discards any result resolved by core function promises: only the updated results from the plugin pipes chain are used (this only applies to events emitted by API functions obviously)
  • introduce a new type of events, ask:
    • ask events are the reverse of pipe ones: if pipes can be listened by many functions and emitted at only 1 place, ask events can only be listened by 0 or 1 function, but may be emitted at many places in the code

Pipes listenable by core functions and the new ask events are meant to loosely couple Kuzzle objects: instead of invoking objects directly (and sometimes manipulate their properties from the outside), objects will now be able to submit side-effect to all listening core functions without knowing which ones.
And they will be able to request information without having any knowledge about what object will ultimately answer, or from where the information will come from.

In other words, this PR aims at restoring the principle of single responsibility to objects, freeing them of knowing what side-effect should be applied to what object.
This also introduces modularity, as we'll now be able to replace (or deactivate) parts of the code with ease.

Other changes

  • The timeout on plugin pipes has been removed: it was an artificial timeout, where we could only send an early "timeout" response to users, while being unable to really abort the ongoing pipe running in the event loop. Now, pipes can run as long as they need to, and users will have to resort to traditional monitoring methods to trace potential slowdowns (which is a good thing)

Optimizations

  • Instead of creating timers everytime a plugin pipe was executing, to emit warnings when they take too long, a simple timestamp calculation is done at the end of pipe executions, eventually logging a warning if the pipe took too long. This prevents creating/destroying timers at high frequences, and this is more useful when reading logs as the pipe execution time is now logged

@codecov
Copy link

codecov bot commented May 1, 2020

Codecov Report

Merging #1613 into 2-dev will increase coverage by 0.03%.
The diff coverage is 97.04%.

Impacted file tree graph

@@            Coverage Diff             @@
##            2-dev    #1613      +/-   ##
==========================================
+ Coverage   93.31%   93.34%   +0.03%     
==========================================
  Files         104      105       +1     
  Lines        7060     7082      +22     
==========================================
+ Hits         6588     6611      +23     
+ Misses        472      471       -1     
Impacted Files Coverage Δ
lib/core/plugins/manager.js 88.71% <91.37%> (-0.95%) ⬇️
lib/core/events/emitter.js 100.00% <100.00%> (ø)
lib/core/events/pipeRunner.js 100.00% <100.00%> (ø)
lib/core/events/waterfall.js 100.00% <100.00%> (ø)
lib/kuzzle.js 86.46% <100.00%> (-2.02%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca14555...081b523. Read the comment docs.

Copy link
Contributor

@Aschen Aschen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍
(Also, I just noticed that the GatewayTimeoutError is not used anymore)

default.config.js Show resolved Hide resolved
lib/core/events/emitter.js Show resolved Hide resolved
lib/core/events/emitter.js Outdated Show resolved Hide resolved
lib/core/events/emitter.js Outdated Show resolved Hide resolved
lib/core/plugins/manager.js Outdated Show resolved Hide resolved
doc/2/plugins/guides/pipes/index.md Show resolved Hide resolved
@scottinet
Copy link
Contributor Author

About the GatewayTimeoutError object, it has still its uses. In plugins for instance, but it could be useful in Kuzzle later. So, I'd say we should keep it and not bother about it being used or not.

@scottinet
Copy link
Contributor Author

@Aschen > I just added a commit, making ask events trigger their associated hooks when invoked.
As discussed, this enable objects to simply listen to these interactions without any intention to respond.

Use cases: logs, statistics... and cluster synchronization.

@sonarcloud
Copy link

sonarcloud bot commented May 5, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.1% 0.1% Duplication

@scottinet scottinet merged commit 5bbf537 into 2-dev May 12, 2020
@scottinet scottinet deleted the internal-pipe-and-ask-events branch May 12, 2020 11:21
This was referenced May 15, 2020
Aschen added a commit that referenced this pull request May 15, 2020
# [2.2.0](https://github.com/kuzzleio/kuzzle/releases/tag/2.2.0) (2020-05-15)


#### Bug fixes

- [ [#1626](#1626) ] Returns a token expired error when using expired jwt   ([Aschen](https://github.com/Aschen))
- [ [#1623](#1623) ] Fix refresh propagation for document:delete   ([Aschen](https://github.com/Aschen))
- [ [#1611](#1611) ] Create user before credentials   ([benoitvidis](https://github.com/benoitvidis))
- [ [#1596](#1596) ] Keep the number of opened scroll contexts in check   ([scottinet](https://github.com/scottinet))

#### New features

- [ [#1614](#1614) ] Add bulk:deleteByQuery   ([Aschen](https://github.com/Aschen))

#### Enhancements

- [ [#1613](#1613) ] Events overhaul: loose coupling + plugin pipe timeout removal   ([scottinet](https://github.com/scottinet))
- [ [#1617](#1617) ] Allows inter-plugins method calls with side-effect   ([Aschen](https://github.com/Aschen))
- [ [#1618](#1618) ] Add user ID in forbidden error log   ([Aschen](https://github.com/Aschen))
- [ [#1599](#1599) ] Prevent user overriding with admin:loadSecurities   ([Aschen](https://github.com/Aschen))
- [ [#1606](#1606) ] Lighter and faster MQTT server   ([scottinet](https://github.com/scottinet))
- [ [#1598](#1598) ] Add token hash in API key   ([Aschen](https://github.com/Aschen))
- [ [#1602](#1602) ] Automatically index new fields after collection:update   ([Yoann-Abbes](https://github.com/Yoann-Abbes))
---
scottinet added a commit that referenced this pull request May 21, 2020
…1631)

# Description

This is an emergency patch, fixing a behavior broken by #1613 
Plugins can attach either functions or function names to pipe events. 

When using function names, plugin developpers have no way of controlling the context used to invoke their functions, and Kuzzle expects the function to exist within the plugin main object (this is verified with assertions when plugins are loaded by Kuzzle).

With Kuzzle 2.2.0, plugin pipes defined using function names are invoked with an undefined context, breaking existing implementations.

This PR fixes that by binding the plugin pipe function to its object of origin, when loading the plugin. 

This is only applied to plugin pipes defined with function names: when using functions, plugin developpers are responsible of the function's context, as Kuzzle has no way of knowing in what object the function belongs.

# How to test

Load a plugin with a pipe function declared using a function name, and use `this` in the function.

Example:

```js
'use strict';

class Foo {
  constructor() {
    this.foo = 'bar';
  }

  async init (config, context) {
    this.config = config;
    this.context = context;
    this.pipes = {
      'server:beforeNow': '_lol'
    };
  }

  async _lol (rq) {
    console.log('============= LOL: ', this.foo);
    return rq;
  }
}

module.exports = Foo;
```

Without this fix, this fails with a `PluginImplementationError` stating that the `foo` property cannot be read from `undefined`.
With the fix, this pipe runs as expected.

# Other changes

* Simplify unit tests a bit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants