Support backend-only development mode (#1327)

[dominikriemer]

Purpose

Modify the backend environment which allows to run StreamPipes for backend-only development purposes.
Added a new service definition ui-custom-conf which adds a volume mapping from a custom Nginx default.conf file where the upstream of the reverse proxy can be set to a hostname that links to the locally running backend.

Remarks

PR introduces (a) breaking change(s): no
PR introduces (a) deprecation(s): no

[dominikriemer]

@RobertIndie I had a look at the backend-only configuration and think this setup has never worked as the UI could not properly connect to the locally running backend.
The backend environment now includes a new service ui-custom-conf where a volume mapping is included that links to a default.conf file that can be customized in the deploy/standalone/ui-custom-conf folder. I set this to host.docker.internal which should work on Windows and Mac systems but not under Linux (there the local IP needs to be set that can be resolved from the Docker container where the UI is running).

I also had some problems when trying to create adapters and pipelines with the extensions service running in Docker as more environment variables need to be set in order to make sure the extensions get the right broker connection settings from the backend. So I removed the extensions-all-jvm service and this would need to be started locally as well in this mode.

Sorry for the long explanation - can you please give it a try? I hope this workaround works ;-)

[RobertIndie]

@dominikriemer Hi, Thanks for your PR. The UI works now!

But when I start all-extensions-jvm locally, it raises these errors:

2023-02-22T22:34:56.060+08:00 ERROR 70202 --- [           main] o.a.s.s.e.connect.ConnectRestClient      : Could not register adapter at url - is a 'StreamPipes Core' service running?

org.apache.streampipes.commons.exceptions.SpRuntimeException: Could not connect to the StreamPipes API - please check that StreamPipes is available
	at org.apache.streampipes.client.http.HttpRequest.executeRequest(HttpRequest.java:111)
	at org.apache.streampipes.client.api.AbstractClientApi.post(AbstractClientApi.java:46)
	at org.apache.streampipes.client.api.AdminApi.registerAdapters(AdminApi.java:36)
	at org.apache.streampipes.service.extensions.connect.ConnectRestClient.register(ConnectRestClient.java:38)
	at org.apache.streampipes.service.extensions.connect.ConnectWorkerRegistrationService.registerWorker(ConnectWorkerRegistrationService.java:35)
	at org.apache.streampipes.service.extensions.ExtensionsModelSubmitter.afterServiceRegistered(ExtensionsModelSubmitter.java:53)
	at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.startExtensionsService(StreamPipesExtensionsServiceBase.java:77)
	at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.init(StreamPipesExtensionsServiceBase.java:54)
	at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.init(StreamPipesExtensionsServiceBase.java:44)
	at org.apache.streampipes.extensions.all.jvm.AllExtensionsInit.main(AllExtensionsInit.java:38)
Caused by: org.apache.http.client.ClientProtocolException: null
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.apache.http.client.fluent.Request.internalExecute(Request.java:173)
	at org.apache.http.client.fluent.Request.execute(Request.java:177)
	at org.apache.streampipes.client.http.HttpRequest.executeRequest(HttpRequest.java:98)
	... 9 common frames omitted
Caused by: org.apache.http.ProtocolException: Target host is not specified
	at org.apache.http.impl.conn.DefaultRoutePlanner.determineRoute(DefaultRoutePlanner.java:71)
	at org.apache.http.impl.client.InternalHttpClient.determineRoute(InternalHttpClient.java:125)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
	... 14 common frames omitted

Do I miss something? Semes the target host is not specified. How can I set the host?

[dominikriemer]

Great!
Can you check which IP or hostname is assigned to the backend upon start (there should be some logging info right at the beginning). The extensions-all-jvm service should receive the hostname of the backend from Consul and use this host to connect to the backend.

[RobertIndie]

Hi @dominikriemer
I restart all my services and it all works now! Thank you. This PR is awesome.

[RobertIndie]

Should we cleanup this comment?

[dominikriemer]

I removed the comment and will add the info to the documentation, thanks!