memory of interactions

[sandro-pasquali]

Are there any best practices for managing many users entering and exiting independent conversations? For example, given the 'knock knock' example, what is the best way to provide that conversation to thousands of concurrent users (remembering each users previous response, etc).

User vars seems to be the answer, but that won't work when there are say a million concurrent users -- memory, redundancy, etc.

Sorry if this is the wrong place for this kind of question...

[kirsle]

User vars are the way to go, but you're right that it doesn't scale indefinitely for millions of users. It should still get you pretty far though (I think most bots would store less than a few KB of data per user depending on how many variables you use in your bot).

Systems to replace the in-memory user variable storage can't currently be implemented in a JavaScript-friendly (read: asynchronous) manner. See previous discussions about this. In the Python version of RiveScript, for example, the user session manager can be swapped out with a third party implementation, with an example using Redis, but Python doesn't have the expectation that all things should be done async so it was more possible here than in the JavaScript version.

It might be less efficient but you could use the same "username" for all requests and just replace the user vars each time, e.g. with setUservars() and clearUservars(), so the maximum amount of memory needed per request is equal to the maximum data you'd store for an individual user. But it'd be at the expense of a lot of memory allocations. JavaScript is pretty fast at that anyway, though. 😉

[sandro-pasquali]

@kirsle Thanks. The set/unset solution is something I'd come to as well, which is a little "clumsy", but could be made to work. The pipeline of request->getVarsFromStore->setVarsInRive->execute->getVarsFromRive->setVarsInStore->clearVarsInRive at least makes sense.

Another approach might be schizophrenia -- one brain for common tasks (same for all users) and another describing conversation trees requiring something like the above per-user implementation.

Also, if I just move everything into function calls, I can just do the work of storing vars myself any way I like in the server context, and managing reentrancy myself. Then Rive is just used as a parser and a stateless dispatcher to the next instruction (ie. function call).

[davidchriqui]

What about scaling horizontally + uservars persistence (cf eg/ directory)?

[kirsle]

I'd probably have a background thread (maybe on a setInterval() loop), that every few seconds or so, calls getUservars() and puts them in a persistent backend store, like Postgres or Mongo. For scaling horizontally, something super fast like a Redis instance might be shared among the servers. I'd try to get "sticky sessions" where the same user ends up talking to the same server, but if that wouldn't work, use Redis as an intermediary cache that all the servers get user variables from and have another process periodically put everything from Redis into your database.

[sandro-pasquali]

I came across this library: https://github.com/CapacitorSet/rebridge

It might be interesting to simply proxy an object to redis in this way, since it would likely require few changes to your code (you'd just proxy whatever object you are referencing now, and the dot-access stays the same). FWIW.