Plugin support #1396

andremedeiros · 2020-04-16T00:59:22Z

Fixes #1353

To do:

Not fixing (see rationale below):

Load plugins list from environment variable
Improve plugin loading error messages

CLAassistant · 2020-04-16T00:59:29Z

All committers have signed the CLA.

andremedeiros · 2020-04-16T01:27:31Z

cmd/common.go

@@ -104,6 +104,14 @@ func getNullString(flags *pflag.FlagSet, key string) null.String {
 	return null.NewString(v, flags.Changed(key))
 }

+func getStringSlice(flags *pflag.FlagSet, key string) []string {


I wasn't sure if this was really necessary, as an empty slice is fine. I just wrote it so we could have a panic error handler 😃

andremedeiros · 2020-04-16T01:28:59Z

cmd/run.go

@@ -107,6 +109,34 @@ a commandline interface for interacting with it.`,
 			Left:  func() string { return "    init" },
 		}

+		// Load plugins


Moving this up from where it was.

Long story short: plugins need to be loaded before the runner gets initialized, as lookup paths are going to be resolved by that time. getConfig is the earliest point at which we have access to the plugin parameters so I decided to pull it up and do it here.

As I mention in the other comment, you shouldn't use getConfig() here, but rather move and use getRuntimeOptions()

andremedeiros · 2020-04-16T01:31:54Z

cmd/run.go

+
+		plugins := []plugin.JavaScriptPlugin{}
+		for _, pluginPath := range cliConf.Plugins {
+			jsPlugin, err := lib.LoadJavaScriptPlugin(pluginPath)


I'm not super happy about this way to interface with plugins. This assumes (correctly, for now) that plugins are only for JavaScript.

Ultimately, I think I want to have plugins that export many different types of plugins (say, a metrics plugin that exposes the API to add metrics, but also the metrics itself to export from K6) but I'm not 100% sure of what that looks like or how it's organized. I think this needs to be part of a design discussion where we draw boxes and arrows and the plugin writing UX needs to be heavily considered too.

I agree that some kind of whiteboard session might be beneficial for this.

Further, I really think we need to solve, or at least discuss, this before this can get merged, or we'll end up in a situation where it will get increasingly hard to change for every plugin that's written. cc @na- Scratch that, apparently the scope for this was js plugins only at this point.

andremedeiros · 2020-04-16T01:32:17Z

cmd/run.go

+
+			// Run plugin setup and add it to the module tree
+			if err = jsPlugin.Setup(); err != nil {
+				return err


I'm not sure we should just return without tearing down any other potential plugins that succeeded. Might be a TODO.

You could've possibly defered the teardown as it is currently run at the end of this same function ... which definitely needs some refactoring ...

andremedeiros · 2020-04-16T01:32:52Z

cmd/run.go

+
+			// TODO: does this belong here, or is it the module package's responsibility?
+			mods := jsPlugin.GetModules()
+			for path, api := range mods {


This probably needs to be a method in the modules package, as the runner shouldn't care about this sort of thing.

Changed it so that it's a method in the modules package. Feels much nicer there.

andremedeiros · 2020-04-16T01:33:46Z

cmd/run.go

@@ -473,6 +499,12 @@ a commandline interface for interacting with it.`,
 			}
 		}

+		// Teardown plugins
+		for _, jsPlugin := range plugins {


Do we care if teardowns error out? Should we log after the output or are there repercussions for that? (ie. any tools that might start failing because output is slightly different)

definitely log it

Yes, please log any errors here

js/initcontext.go

andremedeiros · 2020-04-16T01:36:29Z

lib/options.go

@@ -285,6 +285,9 @@ type Options struct {

 	// Redirect console logging to a file
 	ConsoleOutput null.String `json:"-" envconfig:"K6_CONSOLE_OUTPUT"`
+
+	// Plugins to load
+	Plugins []string `json:"plugins" envconfig:"K6_PLUGINS"`


Still need to figure out how this envconfig flag works. I couldn't get it to read from the environment but I'm probably just missing something really basic.

envconfig has issues (#671) and we want to replace it, but the problem here is not related to them... The reason you can't read from the environment is that this reading happens after you've already initialized the plugins. The "consolidated" version of lib.Options is constructed here: https://github.com/andremedeiros/k6/blob/9dbc2f9ff3d7dbceeb6f06c58b81dcfda3fb3243/cmd/run.go#L169-L172

And, as you can see, this "consolidation" consists of combining the {option defaults + JSON config options + the script exported options + environment variables + CLI flags}, in increasing order of precedence (more info). That chicken-and-egg situation is why you literally can't fix this here, given that plugins would have to be available when the init context of the script is executed to get its exported options.

Instead of lib.Options, you should have the plugin list in RuntimeOptions, which contain things that might affect even the initial script execution (and thus, the exported script options), like environment variables. They can also easily be moved earlier in the cmd/run.go execution as well: https://github.com/loadimpact/k6/blob/0e94653c9954c17b2b1a480b456e23ec120feff2/cmd/run.go#L123-L126

andremedeiros · 2020-04-16T01:36:55Z

lib/plugin.go

+	"github.com/loadimpact/k6/plugin"
+)
+
+func LoadJavaScriptPlugin(path string) (plugin.JavaScriptPlugin, error) {


andremedeiros · 2020-04-16T01:38:44Z

lib/plugin.go

+	return jsPlugin, nil
+}
+
+func loadPlugin(path string) (*goplugin.Plugin, error) {


I want to expand on this (maybe not necessarily for v1) to ensure that we return nice error messages for things like:

plugin built with a different version of the plugin package

plugin built with a different version of go

file not found

wrong architecture

andremedeiros · 2020-04-16T01:39:00Z

plugin/plugin.go

+
+package plugin
+
+// A Plugin ...


I'm really bad at writing docs. Can you tell?

Maybe Plugin is used to extend the javascript VM with functionality implemented in go.?

codecov-io · 2020-04-16T01:39:17Z

Codecov Report

Merging #1396 into master will increase coverage by 0.03%.
The diff coverage is 23.40%.

@@            Coverage Diff             @@
##           master    #1396      +/-   ##
==========================================
+ Coverage   75.22%   75.26%   +0.03%     
==========================================
  Files         150      151       +1     
  Lines       10911    10954      +43     
==========================================
+ Hits         8208     8244      +36     
- Misses       2238     2241       +3     
- Partials      465      469       +4

Impacted Files	Coverage Δ
cmd/run.go	`8.88% <0.00%> (-0.40%)`	⬇️
lib/options.go	`93.14% <ø> (ø)`
lib/plugin.go	`0.00% <0.00%> (ø)`
cmd/common.go	`42.50% <60.00%> (+2.50%)`	⬆️
js/initcontext.go	`88.00% <66.66%> (-2.22%)`	⬇️
cmd/options.go	`71.09% <100.00%> (+0.45%)`	⬆️
loader/loader.go	`86.42% <0.00%> (+1.42%)`	⬆️
loader/cdnjs.go	`85.71% <0.00%> (+85.71%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9586f2f...9dbc2f9. Read the comment docs.

simskij

Thank you for this! Here are some initial thoughts. I will do a more thorough review later when there are tests and such.

simskij · 2020-04-16T10:12:30Z

plugin/plugin.go

+
+package plugin
+
+// A Plugin ...


Maybe Plugin is used to extend the javascript VM with functionality implemented in go.?

js/initcontext.go

simskij · 2020-04-16T10:34:26Z

cmd/run.go

+
+		plugins := []plugin.JavaScriptPlugin{}
+		for _, pluginPath := range cliConf.Plugins {
+			jsPlugin, err := lib.LoadJavaScriptPlugin(pluginPath)


I agree that some kind of whiteboard session might be beneficial for this.

Further, I really think we need to solve, or at least discuss, this before this can get merged, or we'll end up in a situation where it will get increasingly hard to change for every plugin that's written. cc @na- Scratch that, apparently the scope for this was js plugins only at this point.

andremedeiros · 2020-04-17T00:06:11Z

I'm making the choice to remove support to specify plugins via K6_PLUGINS. Here's why.

When run gets invoked, there are a few things that happen:

cliOpts are gathered (these don't care about env config)
runtimeOptions are gathered
the JavaScript code that will get executed gets loaded up and parsed
the consolidated config (which does include environment) is gathered

Now, an easy fix would be to switch 3 and 4, but 4 depends on 3 to help gather options. What's more, at the time of compilation, require statements get resolved, so plugins already have to be loaded.

I could try to refactor this dependency out, but that would require making product decisions that I'm not equipped to make. I'm happy to take over this work after a whiteboarding session with the team in a future PR, if you think it makes sense.

andremedeiros · 2020-04-17T00:37:51Z

With regards to nicer errors, unfortunately the plugin library does not expose error properties I can match against. Matching against error strings is heavily discouraged, as those can change anytime, so I won't be adding nicer error messages. Instead, errors are shown as they come, wrapped to help indicate what the source of the error is.

mstoykov · 2020-04-21T09:53:05Z

.circleci/config.yml

+      - run:
+          name: Build plugin artifact for tests
+          working_directory: /tmp
+          command: |
+            go version
+            export GOMAXPROCS=2
+            export PATH=$GOPATH/bin:$PATH
+            go get -d github.com/andremedeiros/leftpad
+            go build -buildmode=plugin -o /tmp/leftpad.so github.com/andremedeiros/leftpad


I do think that:

the test should work without an additional step before that of building the plugin ... this should probably happen in the test ?

I would really prefer if the test plugin is in our repo - probably in the samples folder? (great plugin name ;) )

great plugin name

I can't take the credit. Someone in the slack suggested that for a test plugin, and I just went with it :-P

mstoykov · 2020-04-21T10:02:30Z

js/modules/index.go

+	for path, impl := range modules {
+		importPath := fmt.Sprintf("k6-plugin/%s", path)
+		PluginIndex[importPath] = impl
+	}


nitpick: I do think it will probably be easier to strip the prefix in the lookup ... although maybe the error message will need some work so it does actually include the "k6-plugin/"

mstoykov

Sorry for the delay :( . I did take a quick look - looks good to me, but I will need a second deeper look when I have more time. Thanks for doing it :D

The things that I do think will definitely need fixing are :

the test will need to work without running some other commands before that ...
the example(sample) plugin will need to live inside the k6 repo.
we need to error out on windows with a nice error (maybe this can be left for us if you don't have a windows machine to test it on), also the test needs to be skipped on windows :)

I have other thoughts on some types, module names and where things are put, but this is dependant on how much we want to try to design now ...

Should I understand that this works for you? Like, have you used it for your use case, successfully?
Again sorry for the slow reviews ... they won't get much faster soon unfortunately and thanks :D

andremedeiros · 2020-04-21T12:57:02Z

Sorry for the delay :( . I did take a quick look

No apologies necessary. I appreciate you taking a look!

the test will need to work without running some other commands before that ...

To be clear, are you OK with me exec'ing go and building things from a test? I wasn't sure if that was kosher or not, but I'm happy to get that done inside the tests.

the example(sample) plugin will need to live inside the k6 repo.

sample/ is a perfect name. On it :-)

we need to error out on windows with a nice error (maybe this can be left for us if you don't have a windows machine to test it on), also the test needs to be skipped on windows :)

Correct. I ignored this as something I was going to tackle later but I'll get on it.

I have other thoughts on some types, module names and where things are put, but this is dependant on how much we want to try to design now ...

Happy to hear them. We can either discuss here or have a call where we draw boxes and arrows and update the issue / code. I want this to work for you too, so whatever it takes to get it right the first time.

Should I understand that this works for you? Like, have you used it for your use case, successfully?

It works for me, yeah! We're loading a plugin and running tests with it, so that's a huge success!

imiric

Nice work on this, I was imagining it would be much more complex. I agree with @mstoykov's comments about moving the sample plugin for tests in the repo, and removing the Makefile changes. Not sure if it could be compiled with a go:generate declaration, or if there's a cleaner alternative to exec-ing go build in the test, but it's something to look into. You could probably avoid this by injecting mock plugins in tests and have more simpler lower-level tests for different scenarios, and just a few high-level ones that require compilation.

imiric · 2020-04-21T14:08:25Z

cmd/run.go

+			}
+
+			// Run plugin setup and add it to the module tree
+			if err = jsPlugin.Setup(); err != nil {


We might want to consider running this in a separate goroutine with some timeout, to avoid a misbehaving plugin from blocking further execution, and maybe load them concurrently before the Runner is created.

We might want to consider running this in a separate goroutine with some timeout

Added to the list :-)

andremedeiros · 2020-04-21T16:40:20Z

Nice work on this, I was imagining it would be much more complex.

Thank you, but now you're making me think I might have missed something :-P

Not sure if it could be compiled with a go:generate declaration, or if there's a cleaner alternative to exec-ing go build in the test, but it's something to look into.

go generate is a mechanism to populate go code, but it won't turn go code into executable / linkable artifacts. I'm afraid os.Exec might be the only way to go here.

You could probably avoid this by injecting mock plugins in tests

I feel that we'd get no added value from mocking if we load a real plugin, but ultimately I'll defer the decision to you.

I'll give this another pass tomorrow. Sorry it's taking so long, but I'm glad we're getting something polished out there.

mstoykov · 2020-04-27T12:18:58Z

hi @andremedeiros , the internal discussion more or less finished with "we are unlikely to do any releases before we merge 1007 and given its size it is probably better if we don't add any more changes, also we want it to be done as fast as possible".
Given that this is unlikely to take any less then a month (but hopefully not more then two ;) ), don't feel pressured to work on this if you have other stuff. And thanks again for your works so far

Having said that, some words on the actual PR:

I haven't looked into it but you are likely to need os.Exec and just build the plugin during the making of the tests ... I don't see any other way. Arguably we can test that calling something with the appropriate interface works as it is unlikely the plugin package is that broken ;). Dunno .. I guess this will be dependant on how the os.Exec implementation works (if it adds some considerable amount of overhead for example).
I personally think that the other kind of plugins will likely be so much different than trying to design for them now will leave us with strange interfaces ... I would argue all different parts with plugins will just have their own interface, in the end, either way. Even if they share 3 methods enforcing a common interface seems like it won't help anything. Again I haven't looked all that much in the code so I might be wrong - but given that this is interfaces we can probably fix this in the future.
The change you've done to the configuration parsing are probably breaking something - you should just add this to the RuntimeOptions IMO (where a lot of other options should be either way >.>)

Again thank you for the work, and again sorry we are likely going to merge it anytime soon - we still want it and think it's a great change :D

…n run end

andremedeiros · 2020-05-09T23:51:57Z

Alright, definitely done this time around.

Moved the plugin sample to the repo
Parallelized plugin setup
Added prefixes
Reverted the CI changes

I wanted to get the tests to run with and without race detection, so I had to add a testhelpers/israce package to expose a flag, since Golang doesn't have that built-in. This is so that we can compile the plugin with the same runtime as the test is being run with.

Fixed the Windows implementation and tests too.

Only thing in the way of a fully green build right now is the linter, which is complaining about silly things that we have full control over or want to stay by design.

Again, sorry it took so long -- life kinda got in the way, but we're there now!

mostafa · 2020-05-10T13:13:51Z

Hi @andremedeiros,

I just made a simple PoC on top of your PR: k6-plugin-kafka.

andremedeiros · 2020-05-10T17:05:30Z

Holy shit @mostafa, well done! Got any feedback re: the API and implementation? Would love to hear from the first plugin implementer out there!

mostafa · 2020-05-10T22:16:15Z

Since I am not a gopher, it was a little weird to start with, but with your leftpad example, I was good to go. I think we should do more work on docs and basics of implementing one. Also the API was very straight-forward. 👍

mostafa · 2020-05-12T09:03:02Z

Hi @andremedeiros,

I don't know if it is by design or any other way around, but the memory consumption behavior of the plugin seems to be like JS code, in that it expands with the number of VUs. I expected it not to increase memory usage, which is not the case. I think the goal for writing a plugin, rather than bundling a JS module/package, is to fully/partially address this memory consumption issue.

Any comments?

andremedeiros · 2020-05-12T17:27:02Z

That makes sense to me.

I assume that every VU has its own JS VM instance running, and that would get copied/cloned from the original. It wouldn't be more than, say, loading pre-built modules tho.

mostafa · 2020-05-24T10:18:22Z

Hi @andremedeiros,

I am working on sending metrics to the k6 process from inside the plugin, yet I've hit the wall of state being nil. AFAIK, The ctx variable is not null, but the GetState doesn't happen to be able to dereference the struct. I don't know if it is related to the way the plugin support is implemented or the way the context object is passed to the functions via Goja.

andremedeiros · 2020-06-24T01:06:44Z

Hi friends! Any chance this gets merged soon?

simskij · 2020-06-24T10:11:52Z

Hi friends! Any chance this gets merged soon?

Hi!

At this point, this PR is still pending #1007 and making sure that the content of that PR is stable.
Thank you for understanding.

Best,
Simme

na-- · 2020-07-20T08:38:34Z

Hey, @andremedeiros 👋 We finally released k6 v0.27.0 last week, so the time for merging this PR has almost arrived! 🎉 We're currently cleaning up some minor issues and preparing to release v0.27.1 in the next few days, but our top priority after that will be adding support for JS plugins! In the meantime, can you rebase this PR on top of the latest master, given how many changes there were?

na--

@andremedeiros, I'm terribly sorry for the super delayed review... 😞 And thank you for the initiative and the awesome PR! It mostly looks very good, the only issue I can see, apart from a couple of nitpicks, is that the plugin list should be in lib.RuntimeOptions and not lib.Options. And the fact that it has to be rebased on the latest master, which will probably result in a few conflicts post-#1007 ... 😞

We're aiming to get this merged in k6 soon, so we can include it in k6 v0.28.0, slated for release mid-September. Given how delayed we were in getting back to you, I don't want to presume that you have the time or energy to get back to it right now... Can you update us on your availability and willingness to rebase this and make the minor fixes in the coming days?

No worries and no pressure if you don't have the time or desire to do so now, just please let us know. Given that it's basically one of our top priorities right now, we can take it from here and finish up the remaining minor things ourselves (of course, giving you full credit in the release notes, commits, etc.).

btw one additional item we discovered we'll have to fix on our own is that, for plugins to work, we'll have to start building k6 with CGO_ENABLED=1 again, so we'll likely also have some CI work regardless

na-- · 2020-08-12T11:04:28Z

cmd/run.go

+			// Do the part that actually takes time in a runner group.
+			g.Go(jsPlugin.Setup)
+
+			modules.RegisterPluginModules(jsPlugin.GetModules())


Hmm we should probably emit an error if there are conflicts in plugin module names?

na-- · 2020-08-12T11:10:45Z

cmd/run.go

@@ -473,6 +499,12 @@ a commandline interface for interacting with it.`,
 			}
 		}

+		// Teardown plugins
+		for _, jsPlugin := range plugins {


Yes, please log any errors here

na-- · 2020-08-12T11:20:52Z

lib/options.go

@@ -285,6 +285,9 @@ type Options struct {

 	// Redirect console logging to a file
 	ConsoleOutput null.String `json:"-" envconfig:"K6_CONSOLE_OUTPUT"`
+
+	// Plugins to load
+	Plugins []string `json:"plugins" envconfig:"K6_PLUGINS"`


envconfig has issues (#671) and we want to replace it, but the problem here is not related to them... The reason you can't read from the environment is that this reading happens after you've already initialized the plugins. The "consolidated" version of lib.Options is constructed here: https://github.com/andremedeiros/k6/blob/9dbc2f9ff3d7dbceeb6f06c58b81dcfda3fb3243/cmd/run.go#L169-L172

And, as you can see, this "consolidation" consists of combining the {option defaults + JSON config options + the script exported options + environment variables + CLI flags}, in increasing order of precedence (more info). That chicken-and-egg situation is why you literally can't fix this here, given that plugins would have to be available when the init context of the script is executed to get its exported options.

Instead of lib.Options, you should have the plugin list in RuntimeOptions, which contain things that might affect even the initial script execution (and thus, the exported script options), like environment variables. They can also easily be moved earlier in the cmd/run.go execution as well: https://github.com/loadimpact/k6/blob/0e94653c9954c17b2b1a480b456e23ec120feff2/cmd/run.go#L123-L126

na-- · 2020-08-12T11:21:53Z

cmd/run.go

@@ -107,6 +109,34 @@ a commandline interface for interacting with it.`,
 			Left:  func() string { return "    init" },
 		}

+		// Load plugins


As I mention in the other comment, you shouldn't use getConfig() here, but rather move and use getRuntimeOptions()

na-- · 2020-08-19T16:23:33Z

For completeness' sake, I'll mention it here as well, in case anyone stumbles here first, before reading the issue. This approach for k6 plugins is probably not the way we'll go forward, at least in the short term. For more information about the issues with Go plugins and why we're exploring alternatives, please read through the discussion following #1353 (comment)

imiric · 2020-11-04T17:19:22Z

Hey @andremedeiros, we've decided to adopt the xcaddy approach for plugins after all (see #1688), so I'll close this PR.

Thanks for your hard work of initially proposing the feature and implementing it in this PR, it's great stuff! Unfortunately the Go plugin approach is not a great fit for k6 right now, and the xcaddy system--while not perfect--will offer a better experience for plugin developers, and that's our priority. :)

andremedeiros commented Apr 16, 2020

View reviewed changes

js/initcontext.go Show resolved Hide resolved

andremedeiros commented Apr 16, 2020

View reviewed changes

andremedeiros mentioned this pull request Apr 16, 2020

Plugin support #1353

Closed

simskij reviewed Apr 16, 2020

View reviewed changes

andremedeiros marked this pull request as ready for review April 17, 2020 00:47

na-- requested review from imiric and mstoykov April 21, 2020 07:10

mstoykov reviewed Apr 21, 2020

View reviewed changes

imiric reviewed Apr 21, 2020

View reviewed changes

Andre Medeiros added 3 commits May 9, 2020 17:10

load plugins specified by the command line parameter

a91a30a

fix plugin load invocation

f6d5963

add plugin to modules; look plugins up on require; teardown plugins o…

f1c38d9

…n run end

Andre Medeiros added 3 commits May 9, 2020 17:49

add build flag to plugin file

b938693

turns out file suffixes dont work as i thought

a43fe5d

add race detector so plugin matches platform

9dbc2f9

andremedeiros requested review from mstoykov, simskij and imiric May 9, 2020 23:49

na-- mentioned this pull request May 11, 2020

Support for writing tests in directly in Go. #1445

Closed

mostafa mentioned this pull request Jun 11, 2020

Stress test kafka stream application #999

Closed

mstoykov mentioned this pull request Jul 13, 2020

Can k6 support socket protocol ? not websocket. #1493

Closed

mostafa mentioned this pull request Jul 21, 2020

Add database support #1019

Closed

na-- added this to the v0.28.0 milestone Jul 22, 2020

na-- requested changes Aug 12, 2020

View reviewed changes

This was referenced Aug 18, 2020

Document plugins grafana/k6-docs#85

Closed

Explain how to add a Go package to k6 #1595

Closed

na-- removed this from the v0.28.0 milestone Sep 8, 2020

na-- mentioned this pull request Oct 12, 2020

SFTP protocol support #1667

Closed

imiric closed this Nov 4, 2020

mostafa mentioned this pull request Jun 29, 2021

Support to consume from all partitions mostafa/xk6-kafka#12

Closed

Plugin support #1396

Plugin support #1396

Conversation

andremedeiros commented Apr 16, 2020 • edited

CLAassistant commented Apr 16, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simskij Apr 16, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Apr 16, 2020 • edited

Codecov Report

simskij left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simskij Apr 16, 2020 • edited

Choose a reason for hiding this comment

andremedeiros commented Apr 17, 2020

andremedeiros commented Apr 17, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mstoykov left a comment • edited

Choose a reason for hiding this comment

andremedeiros commented Apr 21, 2020

imiric left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andremedeiros commented Apr 21, 2020 • edited

mstoykov commented Apr 27, 2020

andremedeiros commented May 9, 2020

mostafa commented May 10, 2020

andremedeiros commented May 10, 2020

mostafa commented May 10, 2020

mostafa commented May 12, 2020

andremedeiros commented May 12, 2020

mostafa commented May 24, 2020 • edited

andremedeiros commented Jun 24, 2020

simskij commented Jun 24, 2020

na-- commented Jul 20, 2020

na-- left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

na-- commented Aug 19, 2020

imiric commented Nov 4, 2020

andremedeiros commented Apr 16, 2020 •

edited

CLAassistant commented Apr 16, 2020 •

edited

simskij Apr 16, 2020 •

edited

codecov-io commented Apr 16, 2020 •

edited

simskij Apr 16, 2020 •

edited

mstoykov left a comment •

edited

andremedeiros commented Apr 21, 2020 •

edited

mostafa commented May 24, 2020 •

edited