[RFC] Reimplement job control patch with libuv

[tarruda]

This is an improved version of the job control patch I sent to vim mailing list a few months ago, and it will be the base for the new plugin architecture. The job module implemented here will be reused for spawning plugins.

The basic difference between plugins and plain jobs started with jobstart is that instead of invoking an auto commands passing the raw data to the vimscript programmer, plugins will have access to Neovim msgpack API directly

This is still a WIP so I'm still going to fix documentation, formatting, etc(this was basically a copy and paste from the patch)

[schmee]

Great to see the new plugin architecture taking form! 🎉

[oni-link]

Looks like the process watcher is not closed.

[justinmk]

Couldn't wait, so I just tested it. It almost works :)

In case it's of any value, I get Abort trap: 6 by the following steps:

Create a simple bash script:

$ echo "sleep 3 && exit 42" > test.sh && chmod u+x test.sh

Do the following in nvim:

:au JobActivity * echom string(v:job_data)
:call jobstart('foo', '/Users/justin/test.sh') 

[tarruda]

@justinmk this last commit should have fixed it. Here's something else you can use to play with it:

let g:nc = jobstart('netcat', 'nc', ['-l', '1234'])
au JobActivity netcat call append(line('$'), v:job_data[1])

Open another terminal and type the following:

nc localhost 1234

and start entering lines. Also calling jobwrite(g:nc, str) should write str to the terminal running the netcat client.

Note that I'm still not handling the redraw issue correctly, for now it's simply calling shell_resized() which is sub-optimal.(Still need to figure out the right functions to call for redrawing only the invalidated columns)

[oni-link]

job_start can fail. Returns 0 in case of error.

[tarruda]

I'm curious, is there some "rule" against usage of goto statements? Many say it's always a bad practice, but personally I think they are a simple way to perform cleanup in a function(Currently using it in f_job_start. One can also extract cleanup code into a separate function like I have done here but I think it's a bit of overkill. Here is an interesting article about the subject.

I think this is something we might want to establish in our code style, as I said I'm ok with gotos for cleanup code.

[justinmk]

@tarruda The dogmatism surrounding anti-goto sentiment is cargo-cult adherence to Dijktra's famous "goto considered harmful" essay. It's still the right tool for C occasionally.

The essay was originally written in the context of trying to convince assembly programmers of the 1970s that function dispatch was not going to kill performance.

One can also extract cleanup code into a separate function like I have done here but I think it's a bit of overkill

Exactly. In languages with private functions (and no forward declaration requirement) and RAII or garbage collection, goto is a code smell. But C doesn't have those things, so AFAIK goto is acceptable in some limited cases.

[tarruda]

@justinmk thanks for the link, interesting read :)

[tarruda]

I'm not satisfied with having to call shell_resized after handling each job event. The problem seems to be that the whole redrawing logic is scattered across edit.c/normal.c(not sure though, I have tried playing with the functions in screen.c but couldn't find anything obvious) and is executed whenever keys are pressed. The shell_resized call can be removed by using the 'events as keys' hack I proposed in #395, this will make vim redraw after asynchronous events more 'naturally'. In any case this is something to be fixed in another PR

Some may be wondering why I'm using job ids instead of opaque pointers(could have made the fields of Job private to the job module for example) and the reason is simple: I couldn't find a way to pass arbitrary pointers to vimscript, the simplest solution was to use integer ids.

I think this is ready for review, but I will only merge it after #476 . After this is merged I can implement the msgpack API hopefully we'll be able to use python plugins.

[justinmk]

@tarruda Playing around with netcat worked beautifully. One question so far: there doesn't seem to be any protection from overwriting an existing job with the same job name:

:jobstart('netcat', 'nc', ['-l', '1234'])
:jobstart('netcat', 'nc', ['-l', '1234'])

For a plugin to avoid collisions like this, it would be useful if jobstart() with, say, an empty string as the first parameter returned the next available job id and registered the job without overwriting an existing job. Would it be a bad idea for au JobActivity 1 to match job id 1? That would probably require disallowing job names starting with a number.

[tarruda]

@justinmk they won't overwrite each other, I think what happened is that the second call to netcat failed because it tried listening on the same port as the first.

When you pass a name to jobstart you are actually saying to Neovim: 'When that job sends you some data, emit an 'event' with that name'. This is similar to the event emitter pattern in javascript, except that I tried to use the autocommand infrastructure for it. You can also listen for a namespace of events like this:

:let srv1_id = jobstart('netcat-server-1', 'nc', ['-l', '9991'])
:let srv2_id = jobstart('netcat-server-2', 'nc', ['-l', '9992'])

au JobActivity netcat-server-* call append(line('$'), 'message from server '.v:job_data[0].': '. v:job_data[1])

[tarruda]

I've updated to use the modifications sent by @philix in #476 but I dont like the way we have to forward declare the Event/Job types to avoid circular dependency problem between the headers. Perhaps we should move typedefs/structs to *_def.h files?

[justinmk]

similar to the event emitter pattern in javascript, except that I tried to use the autocommand infrastructure

From a user interface perspective, I think the design works well, so far.

[tarruda]

From a user interface perspective, I think the design works well, so far.

There's still some things missing:

• Get a list of jobs
• Get detailed information about a job, such a pid
• Notification about when a job exits

But I think it's better to worry about those after the plugin infrastructure is implemented because there might still be some modifications to the job module.

[tarruda]

Rebased/squashed, if no one objects in one hour I will merge it

[justinmk]

Probably should use \"%s\" to be consistent with the existing error messages.

[tarruda]

@justinmk / @oni-link Thanks for the feedback, these last commits should have fixed the issues reported

[oni-link]

Change order of operands, so that we really wait 1s. 20 consecutive jobs, that are not alive anymore at this point, decrease remaining_tries to zero without delay.

[tarruda]

👍