Non-exclusive service and rest API for everyone.

[fulmicoton]

We want indexing and search to be able to coexist in the same process.
We also want to have a rest API for indexing servers too (for monitoring etc.)

Change in usage

• quickwit services run starts both searcher and indexer
• quickwit services run [--service <servicelist>]+ runs a subset of services

Rest, gRPC and service

Regardless of which services is running, the command starts a gRPC server and a Rest server.

However, if the searcher service is not available for instance, the search endpoint will however not be available and return an explicit error message.

Cluster membership is always started.
We will use scuttlebutt to advertise the list of service available for each nodes.

[dzvon]

I have some questions here about this issue.

What is the significance of the searcher service? In my opinion, it was mainly used to start gRPC and REST server (and of course membership service) before. If we keep always start those service (gRPC, REST, membership), then does that means we have to keep running searcher service (i.e. searcher should not be an option in --service)?

However, if the searcher service is not available for instance, the search endpoint will however not be available and return an explicit error message.

I'd also like to know, in which conditions the searcher service will be not available?

[fulmicoton]

@dzvon I suspect you meant SearchService ?

It is a bit tricky because we have two trait called SearchService. The important one is the one in quickwit-search/src/service.rs.
It is a service that is then exposed in gRPC (via a gRPC adapter that does not do much: only small conversion) and in REST.

It is possible to run quickwit as a simple "indexer node" without any search.

In that case, the search service is not loaded.

[dzvon]

Got it.

Originally, I was going to try to do this work, so I had some research. Now that you're already working on this, I'll move on. :)

[fulmicoton]

@dzvon My bad I had forgotten to link my PR. m( _ _ )m

[dzvon]

Don't worry, bro. I can still be somewhat familiar with this project structure. :D @fulmicoton