Scala Client with multiple nodes

[shizeying]

How do I create a distributed es elastic4sknn client

[alexklibisz]

You can use the ElastiknnClient.futureClient method to create a client, see here. This is used extensively in the tests, example here. See docs here for details about how to get the actual dependency.

[shizeying]

I checked the api and found that only the code for the stand-alone mode was found, but the test code for the cluster mode was not found.

[alexklibisz]

I'm not sure if I understand. Can you clarify what does stand-alone mode mean and what does cluster mode mean?

[shizeying]

I think：
stand-alone mode ：One elasticsearh service
cluster mode： three elasticsearh services
but I only see stand-alone mode code in redeme and test

[shizeying]

The api I saw for knn to create a client is like this. I did not find a way to create a knn client for distributed elasticsearch (multiple elasticsaeach)

object ElastiknnClient {

  final case class StrictFailureException(message: String, cause: Throwable = None.orNull) extends RuntimeException(message, cause)

  def futureClient(host: String = "localhost", port: Int = 9200, strictFailure: Boolean = true, timeoutMillis: Int = 30000)(
      implicit ec: ExecutionContext): ElastiknnFutureClient = {
    val rc: RestClient = RestClient
      .builder(new HttpHost(host, port))
      .setRequestConfigCallback(
        (requestConfigBuilder: RequestConfig.Builder) => requestConfigBuilder.setSocketTimeout(timeoutMillis)
      )
      .build()
    val jc: JavaClient = new JavaClient(rc)
    new ElastiknnFutureClient {
      implicit val executor: Executor[Future] = Executor.FutureExecutor(ec)
      implicit val functor: Functor[Future] = Functor.FutureFunctor(ec)
      val elasticClient: ElasticClient = ElasticClient(jc)
      override def execute[T, U](req: T)(implicit handler: Handler[T, U], manifest: Manifest[U]): Future[Response[U]] = {
        val future: Future[Response[U]] = elasticClient.execute(req)
        if (strictFailure) future.flatMap { res =>
          checkResponse(req, res) match {
            case Left(ex) => Future.failed(ex)
            case Right(_) => Future.successful(res)
          }
        } else future
      }

      override def toString: String = s"${ElastiknnClient.getClass.getSimpleName} connected to $host:$port"
      override def close(): Unit = elasticClient.close()
    }
  }

[alexklibisz]

I think I understand that you want to have a way to use several URLs, e.g. localhost:9200,localhost:9201,localhost:9202? That should be possible, it's just not exposed in the current API. I've typically followed the pattern of connecting to a single client node or a single URL, which proxies and load balances requests to several search nodes. I can look at doing it later this week or weekend, or I'm happy to look at a PR.

[shizeying]

This kind of creation method is needed, eg: localhost:9200,localhost:9201,localhost:9202, if possible, can it support distributed creation from version 7.9.x?

[alexklibisz]

The client version will still be 7.11.x.y, but the client is compatible with older ES versions, so it will support talking to an older server.

[shizeying]

Does 7.11 already support distributed creation mode?

[alexklibisz]

No I haven't had a chance to work on it yet.

[alexklibisz]

Hi @shizeying I implemented a solution in #252 and released it in version 7.12.0.1. Now you can just create a RestClient, which can include multiple nodes, and pass that to ElastiknnClient.futureClient. Even though the version is 7.12, you can still use this version of the client to communicate with a 7.11 cluster. Let me know if there are any issues with this.