[BUG] Multiple connect servers - One database - minor issue

[mahdib-huma]

Describe the bug
Running multiple Connect servers (connecting to the same database) setting SESSION_STORE to true in docker enviroments variable. Whenever we restart connect servers (either downscale or upscale), Mirth connect admninistrator dashboard doesn't display the correct statistics.

Steps to reproduce the behavior:
1- Run postgres database - for example with helm (helm install db bitnami/postgresql)
2- Run multiple connect servers and connect it to that database - Environmental variable I have used:

-e DATABASE_USERNAME='postgres' -e DATABASE_URL='jdbc:postgresql://host.docker.internal:5432/postgres' -e DATABASE_PASSWORD='password' -e DATABASE='postgres' -e SESSION_STORE="true"
`
3- Run Mirth Connect administrator launcher and connect to the instance then send some messages through a channel (statistics changes)
4- Restart the server by upscale or downscale number of instances
4- Run Mirth Connect administrator launcher again and you will see the statistics are empty. However, if we check database we can see they are stored properly.

Expected behavior
Show statistics properly even if we restart the pods.

Environment (please complete the following information):

• Connect Version -> docker image 4.1.1 (latest)

Additional context

[pacmano1]

Is this bug report with the clustering plugin installed? Curious really if another user maybe can shed some light on this if they use the clustering plugin.

[mahdib-huma]

Is this bug report with the clustering plugin installed? Curious really if another user maybe can shed some light on this if they use the clustering plugin.

I didn't installed any plugins. Just the base docker image with an external postgres database while setting SESSION_STORE="true".

This must be a cache problem or something. Because the statistics displayed in the dashboard are zero, meanwhile in database it is correct. Also the events are displayed correctly. The only issue are statistics in the main dashboard.

[jonbartels]

Running multiple Connect servers (connecting to the same database)

As far as I know this isn't supported. The basic problem is that many tables (like statistics and configuration) use the MC server-id. I think what you might find is that your new instances server ID has changed, the appdata/server.id file is generated on a cold start. I think you can pass it as an environment variable to containerized instances and entrypoint.sh will write it out. I was wrong about this you have to update the sever.id file https://github.com/nextgenhealthcare/connect-docker#the-appdata-folder-

If you look at your database you see that the stats tables (d_msXX) have a server_id column - https://github.com/kayyagari/connect/blob/je/mc-integ-tests/mc-db-tables.png

Same for some of the message content and metadata tables too.

[jonbartels]

Also note that the clustering plugin from NextGen is a paid product. It may not be in NextGens interest to make it too easy to cluster the open source Mirth Connect. The clustering plugin gets poor reviews though and I expect the community would be excited to see you work through this experiment.

[mahdib-huma]

Running multiple Connect servers (connecting to the same database)

As far as I know this isn't supported. The basic problem is that many tables (like statistics and configuration) use the MC server-id. I think what you might find is that your new instances server ID has changed, the appdata/server.id file is generated on a cold start. I think you can pass it as an environment variable to containerized instances and entrypoint.sh will write it out.

If you look at your database you see that the stats tables (d_msXX) have a server_id column - https://github.com/kayyagari/connect/blob/je/mc-integ-tests/mc-db-tables.png

Same for some of the message content and metadata tables too.

Thanks @jonbartels for the quick reply.

Your are right. Make sense if the server id changes so the behaviour changes if it is used in tables.
I've just started working with Mirth to see the scalability, I need to investigate into this more. I will definitely come back to you with more on this.

[jonbartels]

nextgenhealthcare/connect-docker#14 related issue from the container image. Would be a good first PR if someone was feeling motivated.

[jonbartels]

I was motivated to hack and needed a break from my paying job nextgenhealthcare/connect-docker#28 . Does this help you @mahdib-huma ? It would let you define the server ID as an environment variable for your deployments.

[mahdib-huma]

I was motivated to hack and needed a break from my paying job nextgenhealthcare/connect-docker#28 . Does this help you @mahdib-huma ? It would let you define the server ID as an environment variable for your deployments.

@jonbartels Thanks for the effort, really appreciate it. Let me check and will get back to you.

[mahdib-huma]

@jonbartels I have tested your pull request here (nextgenhealthcare/connect-docker#28).

I can confirm it does it's job by setting the server.id because now I can see that server.id inside my tables (check the image)

It looks like it fixes this issue also (#5554)

Although, it doesn't fix the issue I have mentioned here. When I go from once instance to three instances of Mirth server, the statistics change whenever I click channels button and switch back to dashboard (which is really weird). In the database the statistics are correct by the way as you can see in the picture.

[mahdib-huma]

You can check this record also to see it more clearly:

Screen.Recording.2022-12-01.at.23.29.20.mov