Setting Solr cloud with Zookeeper for Sitecore

Solr Cloud : 

SolrCloud is designed to provide a highly available, fault tolerant environment for distributing your indexed content and query requests across multiple servers.

It’s a system in which data is organized into multiple pieces, or shards, that can be hosted on multiple machines, with replicas providing redundancy for both scalability and fault tolerance, and a ZooKeeper server that helps manage the overall structure so that both indexing and search requests can be routed properly. For More Info visit SolrCloud.

Here I am going to explain Solrcloud setup (2 solr nodes on different machine) , 3 ZK (on different server) with shard 2.  Solrconfig file is placed on central location.

Prerequisites :

• Zookeeper-3.4.10 –  3
• Solr -5.3.1  – Node 2
• Shard – 2
• Sitecore 8.1 Update-2
• 7-zip (to extract the zookeeper-3.4.10.tar.gz packages)

Zookeeper Setup /machine -1 : 

• Download Zookeeper-3.4.10 and extract packages using 7-zip.
• Place the extracted files under created directory (ZK_Home) on machine -1. Ex : E:\SolrCloudPOC\ZooServer\Zookeeper1
• Go inside ZK_Home\Conf directory and rename zoo_sample.cfg  to zoo.cfg.
• Create a directory data/1 inside conf folder for storing the ZK data.
• The ID identifies each server, so in the case of this first instance, you would create a file /conf/data/1/myid with the content “1” ( myid File should be without extension)
• Edit the zoo.config file and modify the dataDir parameter where would you like to store the ZK data.  dataDir=/SolrCloudPOC/ZooServer/Zookeeper1/conf/data/1
• Add server entries which list all ZK server that run ZK instances.  For server info visit site External ZooKeeper Ensemble                                                                    server.1= zookeeperServer1.com:2888:3888
  server.2= zookeeperServer2.com:2889:3889
  server.3= zookeeperServer3.com:2890:3890
• Zoo.cfg file looks like :                  dataDir=/SolrCloudPOC/ZooServer/Zookeeper1/conf/data/1
  clientPort=2181
  initLimit=5
  syncLimit=2
  server.1= zookeeperServer1.com:2888:3888
  server.2= zookeeperServer2.com:2889:3889
  server.3= zookeeperServer3.com:2890:3890
• Do host entry ex :                                                                                                              133.0.1.1 zookeeperServer1.com  ( First Server, IP should be of first server)
  133.0.1.2 zookeeperServer2.com ( Second server , IP should be of second server)
  133.0.1.3 zookeeperServer3.com ( Second server , IP should be of third server)
• Now open the command prompt in Administrator mode and navigate ZK_HOME/bin directory. Execute the command zkserver.cmd which will start the ZK server. 
• Repeat all the above steps on all server which run the ZK server.

Solr Setup | Machine – 1 : 

• Install the JRE if it is not installed on server and setup the JAVA_HOME and path 
• Download the Solr -5.3.1 zip file and extract it in one directory. Ex -JAVA_DIR E:\SolrCloudPOC\Solr\solr1
• Do ZK host entry ex :                                                                                                              133.0.1.1 zookeeperServer1.com  ( First Server, IP should be of first server)
  133.0.1.2 zookeeperServer2.com ( Second server , IP should be of second server)
  133.0.1.3 zookeeperServer3.com ( Second server , IP should be of third server)
• Copy directory JAVA_DIR\configsets\sample_techproducts_configs\conf  and place it inside JAVA_DIR and rename the folder as SolrConfig. EX : E:\SolrCloudPOC\Solr\SolrConfig
• Generate the schema.xml and place it inside solrconfig folder. For generating the schema.xml see the link : Generate the Solr Schema.xml
• Now start the Solr. Open the command prompt in administrator mode and navigate the folder JAVA_DIR\bin and execute the command. solr start -cloud -p 8988 -z “zookeeperServer1.com:2181,zookeeperServer2.com:2182,zookeeperServer3.com:2183”
• Browse the solr url : http:localhost:8988/solr and verify the Solr dashboard.
• For stopping the Solr you can use command : solr stop -cloud -p 8988
• Repeat above all steps for setting up the solr on machine -2 . For starting the solr on machine 2 use below command on command prompt. solr start -cloud -p 8989 -z “zookeeperServer1.com:2181,zookeeperServer2.com:2182,zookeeperServer3.com:2183

Push SolrConfig on ZooKeeper Server : 

• We want to manage the Solrconfig by zookeeper centrally.
• Login on Solr machine – 1 Server where Solrconfig placed inside folder E:\SolrCloudPOC\Solr\SolrConfig
• Open the command prompt in Administrator mode. change the  directory E:\SolrCloudPOC\Solr\solr1\server\scripts\cloud-scripts and execute the below command.
• zkcli.bat -zkhost “zookeeperServer1.com:2181,zookeeperServer2.com:2182,zookeeperServer3.com:2183” -cmd upconfig -confname solrconfig -confdir E:\SolrCloudPOC\Solr\SolrConfig
• Browse the solr url : http:localhost:8988/solr and verify the SolrConfig files.  

Create the Collection : 

• The Collections API is used to enable you to create, remove, or reload collections, but in the context of SolrCloud you can also use it to create collections with a specific number of shards and replicas etc. For more info visit :  Collection API
• Browse below URL in Solr server inside browser for creating the collection.
• http://localhost:8988/solr/admin/collections?action=CREATE&name=sitecore_analytics_index&numShards=2&replicationFactor=2&maxShardsPerNode=2&createNodeSet=[IP Address Solr1]:8988_solr,[IP Address of Solr2]:8989_solr&collection.configName=solrconfig
• Verify the created collection after browsing the solr URL.
• Verify created shard in File system : 
• Similarly you can create another collection.

Rebuild the Sitecore Index : 

• Modify the Solr config file on CMS environment and point the base address on SolrCloud address. For more info for setting up the SOLR Config visit link Setting up Solr
• EX :
• Login to Siteore CMS server. navigate the control panel and click on Indexing Manger Link.
• Select the index which you want to rebuild it and rebuild the index.