Sticky Sessions (known as Sticky Responders)

[skaluva]

We started using nats server with java client. We have a usecase where we want to maintain sticky sessions. ie, for a user if first request goes to server1, further requests belong to that user should go to that server only. I'm not finding any examples to do that. Is this supported? Can anyone guide on this

[scottf]

The client does not have the concept of session, but does have a concept of connection to a server. Once established it does not change unless the connection is manually dropped or the server it is connected to goes out of service.
Are you making a new connection for every request? I'm assuming we are talking about some web request with the backend connecting to a NATS server. Your backend can keep a connection open and re-use it for multiple request. Making a connection is relatively expensive just considering the networking. Your backend could keep multiple connections open I suppose and map the connection to a session.
I suppose another way, assuming you provide multiple servers as the bootstrap to a connection, once you get a connection, you can query the ServerInfo from the connection, then use the host and port properties of the actual connected server and build a url. You could then use this url by itself for your next connection bootstrap.
Some additional information about your use case might be helpful.

[skaluva]

Thanks @scottf . Let me give more details on my usecase:

I'm talking about actual server the gets the request from nats. We will be having cluster of nats servers and cluster of actual servers for a service. So when client (in my case, some backend X service with multiple servers) make a request to another backend service ( some Y service with multiple servers), lets say with username xyz/xyzsessionkey, request hits one of natsservers and then delegates request to actual Y service server first time randomly right. Next time, if we get another request with same sessionkey(or identifier) in next 10/15mins(for example), i want to land it under same actual Y server from nats-server.

Hope its clear. Let me know if you need further information.

[scottf]

It's not per request. The client makes a connection to a specific random server from the list of servers (bootstrap) you give the connection. You can configure the bootstrap with one or more servers. You can configure it to not be random, meaning it will always try the first server in the list without shuffling the list.
If you have a webserver, your backend can make as many connections to NATS servers as you like. Each JNATS client connection is fixed to a specific server. If you want a web session to stick to a specific connection, then maintain that mapping in your user's session.
But I don't think you should bother? The client does not maintain any state unless you are consuming messages. If that is your use case, then you will have to manually track your webserver session to the connection/consumer.
So for your use case, are you doing a request-reply through NATS, to try to resolve some question for the web request? If you are you, do not care which connection makes the request, since all servers in the cluster will give the same answer.
The only other thing, I suppose is your backend listeners could be designed to listen to specific subjects and only those instances would respond. See next message.

[scottf]

You could make a general request on a subject like "who-can-handle-foo-requests" and each queue listener would respond with a subject, for instance their unique id in the payload "foo-handler-123-session-42"
Maybe there are 7 instances of listeners to "who-can-handle-foo-requests". Each will respond. For basic request-replies, only the very first response will get to you. If you need to see all 7 responses and pick from there you need to do something you would probably just publish a message (versus request-reply) and provide your own "reply-to", i.e. maybe "web-server-789-foo" and your webserver client subscribes to that, get's all responses to "who-can-handle" and decides from there.

The responder server would then listen (subscribe) to the subject they told the web-server-789 that they would listen on, maybe some unique guid ""foo-handler-123-session-42". You can then maintain that subject in your session and always publish to that specific subject. Only that server instance will respond since they are the only one listening.

So a couple things to consider. If you absolutely have to maintain state at the foo handler, I think this is what you need. But if you don't need to maintain state, just make a request reply and let the faster response be your answer, or get all the responses and pick the one you like.

Some examples you can look at are here:
https://github.com/nats-io/java-nats-examples/tree/main/core-reply-request-patterns/src/main/java/io/nats/example

and here:
https://github.com/nats-io/java-nats-examples/tree/main/multi-subject-worker

[skaluva]

@scottf Trying to understand what you said. In my case, we are going to have "Q Group" for those cluster of actual service servers, so that only one server will respond to the request. Here is my question:

1. How i can set request-reply payload with new subject with unique id (to identify some X server responded) , so that callers can send subsequent requests to this new subject?

Also i dont see any API in java library to add dynamically some new endpoint to the service which was bootstrapped already?

Service service1 = new ServiceBuilder()
                .connection(nc)
                .name("Service1")
                .description("Service1 Description") // optional
                .version("0.0.1")
                .addServiceEndpoint(seEcho1)
                .addServiceEndpoint(seSort1A)
                .addServiceEndpoint(seSort1D)
                .build();

[skaluva]

Yes exactly. Any plans to implement it in near future? Also , why there is no addServiceEndpoint API available in Java library after service is build, whereas the same available in Go lang API ?

[scottf]

Regarding the availability of adding endpoints after the service is built and running, this was not a requirement in our internal specification, in fact, I was surprised to find this out as well.

[scottf]

As far as the ability to pin to a server, we don't have a timeline for this.

[skaluva]

Thanks @scottf . I got all answers. As no tentative timeline from you for adding those APIs and providing ability to pin to a server, we will work on extending the current library and modify as per our needs without waiting for new releases. Thank you so much for taking time and clarifying all my queries.

[jnmoyne]

FWIW I think the ability to add or remove an endpoint to an existing service would be a valid feature to add