Enable embedding of gobetween in other processes

[KnicKnic]

There are two ways of going about extending gobetween. One of which is to write a custom discovery plugin and add it for your scenario. However it looks like plugins poll their refresh information. It also means that when I want to update the client code it has to be done in this project (or a fork of it)

Another option is to have a library that is more focused on being driven.

1. Discovery gets pushed externally from library
2. config is not stored in the gobetween library
   1. hosting application scans their DB and replays all loadbalancer rules
3. It would be good if when api returned load balancer is active. (had a way to determine when it was active) I couldn't quickly tell what happens when manager.Create completed
   1. It looked like fully starting the rule was done in a go routine.

Currently I was thinking of extending gobetween by using as a library rather than embedding my version of the discovery and server syncer inside it.

Also the RPC version of the API does not support this paradigm where the caller that is creating the rules will also call to setup set & modify the backend pools.

I would love comments.

The way I envision is you would then layer the gobetween application ontop of this library to help provide backend discovery, RPC, ...

[KnicKnic]

I took a quick crack at allowing backends to be mutable, see KnicKnic@9a7d191

Also I would like to comment that the kubernetes discovery ran into the same issue of backends not being modifiable. #30 (comment)

[KnicKnic]

There are two remaining issues.

1. starting a server with "static" discovery. The initial server.Start calls Discovery.Start , which does not push the backend immediately, and instead launches a go routine to do this. Options:
   1. Create a new backend / option on existing "static" backend whose sole purpose is to shortcut actually doing anything in this go routine, and simply exiting
      1. This would require the user to always use manager.Modify to setup the initial backend pool, and not set them up directly in the create step.
   2. Adding some synchronization primitive such that before manager. Create completes it guarantees that the static discovery got past the initial post of the backends to the scheduler.
   3. Something else?
2. Adding some package that will do the necessary initialization for embedding.

[yyyar]

Hi @KnicKnic
Originally gobetween was designed and build as standalone application. Although it may be useful to have it's core functionality as a library to allow embedding it to other applications, it's was not intended to be used in such manner. It seems doing it properly requires some code reorganization and solving bunch of questions. We didn't seen a demand in it, that's why it's not a priority right now.

Also, there is a difference between extending and embedding: what exactly you want to achieve?
Please provide more info about your use case and what exactly you want to have as a result.

If you need just custom discovery, the easiest way to do it without touching gobetween codebase is to use 'exec' discovery with your own script/program. It may be suitable for most cases. Yep it polls data but it suites most cases we got.

I see it may be needed to push backends update instantly in some cases. It may be implemented, for example, via existing REST, but it's a separate topic and we may create separate issue for this feature.

Thanks!