To provide high availability, automatic fail-over, and load balancing for distributed indexing and search capabilities, the Apache Solr provides the ability to set up a cluster of Solr servers called SolrCloud. Although Solr has embedded Apache ZooKeeper, it's not recommended to approach using this internal ZooKeeper in the production environment because shutting down a Solr instance will also shut down its ZooKeeper instance. In this article, we will learn to set up SolrCloud clusters on windows by setting up external ZooKeeper's ensemble.
Prerequisites:
To set up SolrCloud instance for the Production environment, it is generally recommended to have an odd number of ZooKeeper servers in your ensemble, so the majority of servers are maintained to serve requests. In my case, I have taken 3 servers (solr-vm-1, solr-vm-2, solr-vm-3) to setup SolrCloud clusters.
Steps to follow on each Solr server:
- Add Host Entries
- Setup OpenJDK - Java Runtime Environment
- Install NSSM
- Install ZooKeeper and Configure Ensemble
- Allow ZooKeeper Port in the Firewall
- Verify ZooKeeper is Ready.
- Configure ZooKeeper for SSL
- Install Solr
- Generate and configure Solr certificate
- Configure ZooKeeper endpoint in the Solr configuration
- Create SolrCollection
Step 1. Add Host Entries
for each server, we have to make host entries of all Solr servers including itself for connectivity among each other.
127.0.0.1 solr-vm-1
xxx.x.x.x solr-vm-2
xxx.x.x.x solr-vm-3
On solr-vm-2:
xxx.x.x.x solr-vm-1
127.0.0.1 solr-vm-2
xxx.x.x.x solr-vm-3
On solr-vm-3:
xxx.x.x.x solr-vm-1
xxx.x.x.x solr-vm-2127.0.0.1 solr-vm-3
Step 2. Setup OpenJDK - Java Runtime Environment
- Download the OpenJDK and extracts the downloaded zip file into a directory e.g. JRE
- Set the JAVA_HOME Environment variable and also set up the java.exe folder Path
- Verify the Java installation by running java -version command on Command Prompt or PowerShell
Step 3. Install NSSM Tool - To allow the creation of Windows Services (e.g. Solr, ZooKeeper)
- Downloads the specified NSSM, and extracts it to a directory e.g. Nssm
Step 4. Install and Configure ZooKeeper Ensemble
- Download the Apache ZooKeeper 3.4.14 and extracts it to a directory e.g. zookeeper
- Create ZooKeeper InstanceId - The integer instance number for this Zookeeper node. Must be unique in the current cluster.
- Create a data directory, this is the directory in which ZooKeeper will store data about the cluster. In our case, I have created a folder named "data" in d:\solrcloud\zookeeper
- Create a "myid" file with the contents "1" and put it in the data directory (d:\solrcloud\zookeeper\data) for solr-vm-1, On the solr-vm-2, create a "myid" file with the contents "2", and put it in the data directory (d:\solrcloud\zookeeper\data), On the solr-vm-3, create a "myid" file with the contents "3", and put it in the data directory (d:\solrcloud\zookeeper\data).
solr-vm-1
solr-vm-2
solr-vm-3
- Create a ZooKeeper Config on each server.
- create a file named "zoo.cfg" in D:\solrcloud\zookeeper\conf directory. you can copy and rename the existing file (zoo_sample.cfg) in the same directory.
- The file should have the following information to start and it can be the same on each Solr server.
The clientport, this is the port on which Solr will access ZooKeeper (default 2181). server.1, server2, server.3 are the part of Ensemble Configuration, we need to set these parameters so each node knows who it is in the ensemble and where every other node is. so here our Ensemble Configuration mentioned in the file is configured with hostnames, you can configure it by IP Address as well.
The ports can be any ports you choose, ZooKeeper’s default ports are 2888:3888.
- Install ZooKeeper Service
- Open Command Prompt (run as administrator) and run the below command from the Nssm directory on solr-vm-1 server to install the zookeeper service.
install ZooKeeper-2 and ZooKeeper-3 service like above on solr-vm-2 and solr-vm-3 respectively.
- Start ZooKeeper Service and make sure it is running.
Step 5. Allow ZooKeeper Port's in the Firewall
Allow the ZooKeeper ports (in our case 2971, 2981, 2991) in the windows firewall on each Solr server. Also, create a security rule to allow these port if VM's are on cloud provider - AWS, Azure, etc.
You can allow a port in windows firewall via GUI but can use the below command too to allow it.
New-NetFirewallRule -DisplayName "Allow Zookeeper Inbound" -Direction Inbound -LocalPort 2971 -Protocol TCP -Action Allow
New-NetFirewallRule -DisplayName "Allow Zookeeper Inbound" -Direction Inbound -LocalPort 2981 -Protocol TCP -Action Allow
New-NetFirewallRule -DisplayName "Allow Zookeeper Inbound" -Direction Inbound -LocalPort 2991 -Protocol TCP -Action Allow
DON'T WORRIES :) about the DisplayName, it can be the same. 😊
After the specified port gets opened, you can verify port connectivity between each Solr server using the below PowerShell command
Test-Netconnection solr-vm-2 -port 2971
if you get a True response in TcpTestSucceeded: True then you should be good to move forward.
Step 6. Verify ZooKeeper is Ready.
To verify your Zookeeper server is communicating properly, you can use any TCPClient socket programming code to have a PING/PONG mechanism. Here I have used below Powershell TcpClient code to have a Request with 'ruok' and verify 'imok' Response.
| $zkHost= "solr-vm-1" $zkPort="2971" | ||
| Write-Host "Waiting for ZooKeeper at $($zkHost):$zkPort" | ||
| $client = New-Object System.Net.Sockets.TcpClient | ||
| $sawError = $false | ||
| $isUp = $false | ||
| while($isUp -ne $true) | ||
| { | ||
| try | ||
| { | ||
| $client.Connect($zkHost, $zkPort) | ||
| $ns = [System.Net.Sockets.NetworkStream]$client.GetStream() | ||
| $sendBytes = [System.Text.Encoding]::ASCII.GetBytes("ruok") | ||
| $ns.Write($sendBytes, 0, $sendBytes.Length) | ||
| $buffer = New-Object 'byte[]' 10 | ||
| $bytesRead = $ns.Read($buffer, 0, 10) | ||
| $receivedBytes = New-Object 'byte[]' $bytesRead | ||
| [System.Array]::Copy($buffer, $receivedBytes, $bytesRead) | ||
| $result= [System.Text.Encoding]::ASCII.GetString($receivedBytes) | ||
| if( $result -eq "imok" ) | ||
| { | ||
| $isUp = $true | ||
| if( $sawError -eq $true ) | ||
| { | ||
| Write-Host | ||
| } | ||
| } | ||
| $ns.Dispose() | ||
| } | ||
| catch | ||
| { | ||
| $sawError = $true | ||
| Write-Host "Waiting..." -Newline | ||
| } | ||
| } | ||
| $client.Dispose() | ||
Write-Host "ZooKeeper is up"
|
Like above, verify on other Solr servers too - solr-vm-2 and solr-vm-3.
Step 7. Configure ZooKeeper for SSL
Performs the configuration of a Zookeeper ensemble to use SSL when talking to Solr. Run the below command from any of the Solr servers (doesn't require to run on each zookeeper server.)
cd $solrFolder\server\scripts\cloud-scriptszkCli.bat -zkhost "solr-vm-1:2971,solr-vm-2:2971,solr-vm-3:2971" -cmd clusterprop -name urlScheme -val https
Warning!
Make sure the local java env variable was updated before calling this it will fail if it can't find Java - and it appears to need the path environment set rather than the JAVA_HOME that the rest of Solr requires.
Step 8. Install Solr
- Download the Apache Solr-8.1.1 and extract it to a directory e.g. solr
- Install Solr Service
- Open Command Prompt (run as administrator) and run the below command from the Nssm directory on solr-vm-1 server to install the zookeeper service.
nssm install "ServiceName"
Step 9. Generate and Configure Solr certificate
- Download solrssl.ps1 and before running the Powershell script update script by adding your Solr servers IP Addresses and hostnames details like below. Make sure to include the IP address Or DNS name of Solr Load Balancer.
Once you run the above script you will see two files generated as mentioned in the screen below. Copied both file to SOLR_HOME\server\etc folder and then update solr certificate path into the D:\solrcloud\solr\bin\solr.in.cmd batch file.
- Configure Solr configuration
- Increase Java Min/Max Heap size (as per your needs)
- Update Solr hostname on solr-vm-1, it should solr-vm-2 and solr-vm-3 on other servers
- Update Solr Port and Certificate Path in the Solr.in.cmd file, Make sure the Solr port - 8983 is open at the firewall. You need to install the above generated Solr certificate on -other servers - solr-vm-2, solr-vm-3.
Step 10. Configure ZooKeeper endpoint in the Solr configuration
- Configure the ZooKeeper Ensemble in the Solr.in.cmd file like below, it will same on each Solr server.
- Save Solr.in.cmd file and Start Solr Service and make sure it is running.
- Launch Solr URL - https://solr-vm-1:8983/solr in the browser and verify the things, all three ZK status, graph, etc.
Step 11. Create SolrCollection
- Upload Solr ConfigSet - To create the Solr collection we need two configset - Sitecore and xDB Here are the articles to follow for the creation of both configset.
If we have to make any changes to the Existing configset already uploaded to ZooKeeper we can download the configset and upload it again.
Command to upload configset -
.\solr zk upconfig -d D:\solr-8.1.1\solr-8.1.1\server\solr\configsets\sitecore_configs -n sitecore1 -z localhost:9986
Command to download configset -
.\solr zk downconfig -d D:\solr-8.1.1\solr-8.1.1\server\solr\configsets\sitecore_configs -n sitecore1 -z localhost:9986
- Create Collection - To create the collection we can follow the above link to create it via Solr UI Or can use the Collection API to do that.
$solrCollectionName= "sitecore_core_index"
$shards=1
$replicas=3 $shardsPerNode=1 $solrCollectionConfigSet="Sitecore" $url = "https://localhost:8983/solr/admin/collections?action=CREATE&name=$solrCollectionName&numShards=$shards&replicationFactor=$replicas&maxShardsPerNode=$shardsPerNode&collection.configName=$solrCollectionConfigSet" Invoke-WebRequest -UseBasicParsing -Uri $url | Out-Null
using the above Powershell script we can create the following Sitecore and XDB collections.
- Sitecore Cores -
- "$($collectionPrefix)_core_index",
- "$($collectionPrefix)_master_index",
- "$($collectionPrefix)_web_index"
- "$($collectionPrefix)_marketingdefinitions_master",
- "$($collectionPrefix)_marketingdefinitions_web",
- "$($collectionPrefix)_marketing_asset_index_master",
- "$($collectionPrefix)_marketing_asset_index_web",
- "$($collectionPrefix)_testing_index",
- "$($collectionPrefix)_suggested_test_index",
- "$($collectionPrefix)_fxm_master_index",
- "$($collectionPrefix)_fxm_web_index",
- "$($collectionPrefix)_personalization_index"
- xDBCores -
- "$($collectionPrefix)_xdb",
- "$($collectionPrefix)_xdb_rebuild"
xDB requires to have an Alias of collection, follow links mentioned above to create it from UI.
Once all collections are created, you should see those on the Solr UI.
All Done!
Conclusion:
I hope you have learned something about setting up SolrCoud and ZooKeeper.
Happy Learning! 😊