How to achieve autoscaling of solr service

[sugun999]

Hi,

I have setup solrcloud with below docker compose file.
After that I created collection called "test" with 1 shard and 2 replicationfactor.

Now I would like to scale solr service based on usage load automatically. So that new solr instances will act as replicas and can share read requests load.
I tried with "docker service scale solr=3" but new task/service-instance doesn't get created with my "test" collection.

How can I get this working? Appreciated any help.

version: "3"

networks:
  solr_network:
    external: true

volumes:
  solr:

services:
  solr:
    image: solr:6.6.0
    networks:
      - solr_network
    ports:
      - "8983:8983"
    command: /opt/solr/bin/solr start -f -z zookeeper:2181
    volumes:
      - solr:/opt/solr/server/solr
    environment:
      SOLR_HEAP: 2G
    depends_on:
      - zookeeper
    deploy:
      mode: replicated
      replicas: 2
      placement:
        constraints:
          - node.role == worker
      resources:
        limits:
          cpus: '1'
          memory: 4G
        reservations:
          cpus: '0.5'
          memory: 3G

Thanks.

[makuk66]

Have a look at Overview of SolrCloud Autoscaling in the Reference Guide. Auto-scaling related features were introduced in 7.0, were refined in 7.1, and are being further developed in 7.x.

It sounds like you want to have Solr automatically create a replica of collection when a new node appears. It doesn't do that AIUI; you'll need to manually invoke ADDREPLICA to add the new nodes as replicas to your existing collection.

[beephotography]

Hey there,

I've searched the whole internet for the same question but didn't find any comprehensible answer. I've also read the autoscaling ref but I didn't get it.
With

localhost:8984/solr/admin/collections?action=MODIFYCOLLECTION&collection=gettingstarted&replicationFactor=3

I tried to increase the number of replicas from 2 to 3. When I look into the collection, the replicationFactor got updated indeed. But in the "cloud" view nothing has changed at all.
So if I have to add all additional replicas manually, what's the point in calling the above API?

Maybe somebody could give an example (with all the actions that should be executed) of how autoscaling could be accomplished?

This would be so great!

Thanks

[Martin-Andersen]

I am also trying to add replicas on new nodes. I would like to ask @makuk66 and @sugun999 if the found a solution to the problem.

[shalinmangar]

I have added an answer to the stack overflow question. It is not what you want but I'll try to address these in the next Solr release (Solr 7.5).

[sscholl]

@shalinmangar How is this addressed in Solr 7.5? Thanks!

[makuk66]

@shalinmangar Did something appear in Solr to make this possible through simple configuration? If not let's close this out until such a thing is possible upstream.

[alisade]

This seems to be supported now by Solr 7.6+
check jira links here
https://stackoverflow.com/questions/50839060/solr-autoscaling-add-replicas-on-new-nodes