Sitecore XP 10 Out-of-the-Box: Default Azure PaaS Resources

Sitecore Azure PaaS is a cloud platform that provides a pre-configured and scalable hosting environment for Sitecore applications. As such, it comes with a number of built-in resources that are designed to make it easier to deploy, manage, and maintain Sitecore applications in the cloud.

Here is a list of some of the out-of-the-box resources that come with Sitecore Azure PaaS:

• Azure App Service: A fully managed web app hosting platform that can be used to host Sitecore applications. Sitecore Azure PaaS includes a total of 11 out-of-the-box App Service instances and 6 App Service Plan.

Here is a picturization of which App Service is associated with which App Service Plan

• Azure SQL Database: A managed SQL database service that can be used to store data for Sitecore applications. Sitecore Azure PaaS includes a total of 16 out-of-the-box Databases.

• Azure Cache for Redis: A managed in-memory cache service that can be used to improve the performance of Sitecore applications.

• Azure Service Bus: A messaging service that can be used to enable communication and coordination between decoupled systems and services. Azure Service Bus can be used to send and receive messages between applications, or to build complex workflows using a publish-subscribe model. 

• Azure Application Insights: A performance monitoring service that can be used to detect and diagnose issues in applications. Azure Application Insights can be used to monitor the performance and usage of applications in real-time, and to get alerts when problems arise.

Depending on your specific needs and requirements, you can use some other resources also as mentioned below:

• Azure Application Gateway: A load balancing and web application firewall (WAF) service that can be used to protect and scale web applications.

• Azure SQL Elastic Pool: A service that allows you to pool resources and manage the performance of multiple databases within a single logical server.

• Azure CDN: A content delivery network (CDN) that can be used to deliver static and dynamic content, such as HTML, CSS, JavaScript, and media files, with low latency and high performance.

• Azure SQL Replication: A feature that can be used to replicate data between SQL databases, including for disaster recovery (DR) scenarios.

• Azure Traffic Manager: A load balancing service that can be used to distribute traffic across multiple instances/multiple region of a Sitecore application. Helpful in disaster recovery (DR) scenarios.

• Azure Key Vault: A secure store for secrets, such as passwords, API keys and certificates to associate with Application Gateway Listeners, CDN custom domains, App Service custom domains etc. Also any other secret strings that can be used by Sitecore applications.
  
  
• Azure DNS: A domain name system (DNS) service that can be used to host and manage DNS domains, and to resolve DNS queries from users.
  
  
• Azure Blob Storage: A massively scalable object storage service that can be used to store and retrieve large amounts of data such as App Service Backup, Database Backups, Application Gateway Diagnosis logs, CDN logs etc.

• Azure Monitor: A monitoring service that can be used to collect, analyze, and visualize data from various sources, including Sitecore applications.

These resources can be used to enhance the capabilities and performance of Sitecore applications in the cloud. For example, you can use Azure Application Gateway to protect your application from attacks and to distribute incoming traffic across multiple instances. You can use Azure SQL Elastic Pool to manage the performance and scalability of your database, and Azure CDN to improve the delivery of static content. Azure SQL Replication can be used to ensure that you have a reliable copy of your data in case of a disaster, and Azure DNS can be used to manage your custom domain name and to route traffic to your application.

Thanks for reading, CloudOps ⌂Signing Off! 😊

Configure Solrcloud with External ZooKeeper Ensemble on a Distributed Environment

To provide high availability, automatic fail-over, and load balancing for distributed indexing and search capabilities, the Apache Solr provides the ability to set up a cluster of Solr servers called SolrCloud. Although Solr has embedded Apache ZooKeeper, it's not recommended to approach using this internal ZooKeeper in the production environment because shutting down a Solr instance will also shut down its ZooKeeper instance. In this article, we will learn to set up SolrCloud clusters on windows by setting up external ZooKeeper's ensemble.

Prerequisites:

To set up SolrCloud instance for the Production environment, it is generally recommended to have an odd number of ZooKeeper servers in your ensemble, so the majority of servers are maintained to serve requests. In my case, I have taken 3 servers (solr-vm-1, solr-vm-2, solr-vm-3) to setup SolrCloud clusters.

Steps to follow on each Solr server:

• Add Host Entries
• Setup OpenJDK - Java Runtime Environment
• Install NSSM
• Install ZooKeeper and Configure Ensemble
• Allow ZooKeeper Port in the Firewall
• Verify ZooKeeper is Ready.
• Configure ZooKeeper for SSL
• Install Solr
• Generate and configure Solr certificate
• Configure ZooKeeper endpoint in the Solr configuration
• Create SolrCollection

Step 1. Add Host Entries 

for each server, we have to make host entries of all Solr servers including itself for connectivity among each other.

On solr-vm-1:

127.0.0.1 solr-vm-1
xxx.x.x.x solr-vm-2
xxx.x.x.x solr-vm-3

On solr-vm-2:

xxx.x.x.x solr-vm-1
127.0.0.1 solr-vm-2
xxx.x.x.x solr-vm-3

On solr-vm-3:

xxx.x.x.x solr-vm-1
xxx.x.x.x solr-vm-2

127.0.0.1 solr-vm-3

Step 2. Setup OpenJDK - Java Runtime Environment

• Download the OpenJDK and extracts the downloaded zip file into a directory e.g. JRE

• Set the JAVA_HOME Environment variable and also set up the java.exe folder Path

• Verify the Java installation by running java -version command on Command Prompt or PowerShell

  

Step 3. Install NSSM Tool - To allow the creation of Windows Services (e.g. Solr, ZooKeeper)

• Downloads the specified NSSM, and extracts it to a directory e.g. Nssm

Step 4. Install and Configure ZooKeeper Ensemble

• Download the Apache ZooKeeper 3.4.14 and extracts it to a directory e.g. zookeeper

• Create ZooKeeper InstanceId - The integer instance number for this Zookeeper node. Must be unique in the current cluster.

• Create a data directory, this is the directory in which ZooKeeper will store data about the cluster. In our case, I have created a folder named "data" in d:\solrcloud\zookeeper
• Create a "myid" file with the contents "1" and put it in the data directory (d:\solrcloud\zookeeper\data) for solr-vm-1, On the solr-vm-2, create a "myid" file with the contents "2", and put it in the data directory (d:\solrcloud\zookeeper\data), On the solr-vm-3, create a "myid" file with the contents "3", and put it in the data directory (d:\solrcloud\zookeeper\data).

solr-vm-1

 

solr-vm-2

solr-vm-3

 

 

• Create a ZooKeeper Config on each server.

• create a file named "zoo.cfg" in D:\solrcloud\zookeeper\conf directory. you can copy and rename the existing file (zoo_sample.cfg) in the same directory.
• The file should have the following information to start and it can be the same on each Solr server.

The clientport, this is the port on which Solr will access ZooKeeper (default 2181). server.1, server2, server.3 are the part of Ensemble Configuration, we need to set these parameters so each node knows who it is in the ensemble and where every other node is. so here our Ensemble Configuration mentioned in the file is configured with hostnames, you can configure it by IP Address as well.

The ports can be any ports you choose, ZooKeeper’s default ports are 2888:3888. 

• Install ZooKeeper Service

• Open Command Prompt (run as administrator) and run the below command from the Nssm directory on solr-vm-1 server to install the zookeeper service.

                    nssm install "ServiceName"

 

 

install ZooKeeper-2 and ZooKeeper-3 service like above on solr-vm-2 and solr-vm-3 respectively.

• Start ZooKeeper Service and make sure it is running.

Step 5. Allow ZooKeeper Port's in the Firewall

Allow the ZooKeeper ports (in our case 2971, 2981, 2991) in the windows firewall on each Solr server. Also, create a security rule to allow these port if VM's are on cloud provider - AWS, Azure, etc.

You can allow a port in windows firewall via GUI but can use the below command too to allow it.

New-NetFirewallRule -DisplayName "Allow Zookeeper Inbound" -Direction Inbound -LocalPort 2971 -Protocol TCP -Action Allow

New-NetFirewallRule -DisplayName "Allow Zookeeper Inbound" -Direction Inbound -LocalPort 2981 -Protocol TCP -Action Allow

New-NetFirewallRule -DisplayName "Allow Zookeeper Inbound" -Direction Inbound -LocalPort 2991 -Protocol TCP -Action Allow

DON'T WORRIES :) about the DisplayName, it can be the same. 😊

After the specified port gets opened, you can verify port connectivity between each Solr server using the below PowerShell command

Test-Netconnection solr-vm-2 -port 2971

if you get a True response in TcpTestSucceeded: True then you should be good to move forward.

Step 6. Verify ZooKeeper is Ready.

To verify your Zookeeper server is communicating properly, you can use any TCPClient socket programming code to have a PING/PONG mechanism. Here I have used below Powershell TcpClient code to have a Request with 'ruok' and verify 'imok' Response.

		$zkHost= "solr-vm-1" $zkPort="2971"
		Write-Host "Waiting for ZooKeeper at $($zkHost):$zkPort"
		$client = New-Object System.Net.Sockets.TcpClient
		$sawError = $false
		$isUp = $false
		while($isUp -ne $true)
		{
		try
		{
		$client.Connect($zkHost, $zkPort)
		$ns = [System.Net.Sockets.NetworkStream]$client.GetStream()
		$sendBytes = [System.Text.Encoding]::ASCII.GetBytes("ruok")
		$ns.Write($sendBytes, 0, $sendBytes.Length)
		$buffer = New-Object 'byte[]' 10
		$bytesRead = $ns.Read($buffer, 0, 10)
		$receivedBytes = New-Object 'byte[]' $bytesRead
		[System.Array]::Copy($buffer, $receivedBytes, $bytesRead)
		$result= [System.Text.Encoding]::ASCII.GetString($receivedBytes)
		if( $result -eq "imok" )
		{
		$isUp = $true
		if( $sawError -eq $true )
		{
		Write-Host
		}
		}
		$ns.Dispose()
		}
		catch
		{
		$sawError = $true
		Write-Host "Waiting..." -Newline
		}
		}
		$client.Dispose()
		Write-Host "ZooKeeper is up"

Like above, verify on other Solr servers too - solr-vm-2 and solr-vm-3.

Step 7. Configure ZooKeeper for SSL

Performs the configuration of a Zookeeper ensemble to use SSL when talking to Solr. Run the below command from any of the Solr servers (doesn't require to run on each zookeeper server.)

cd $solrFolder\server\scripts\cloud-scripts

zkCli.bat -zkhost "solr-vm-1:2971,solr-vm-2:2971,solr-vm-3:2971" -cmd clusterprop -name urlScheme -val https

Warning!

Make sure the local java env variable was updated before calling this it will fail if it can't find Java - and it appears to need the path environment set rather than the JAVA_HOME that the rest of Solr requires.

Step 8. Install Solr

• Download the Apache Solr-8.1.1 and extract it to a directory e.g. solr

• Install Solr Service
• Open Command Prompt (run as administrator) and run the below command from the Nssm directory on solr-vm-1 server to install the zookeeper service.

nssm install "ServiceName"

Step 9. Generate and Configure Solr certificate

• Download solrssl.ps1 and before running the Powershell script update script by adding your Solr servers IP Addresses and hostnames details like below. Make sure to include the IP address Or DNS name of Solr Load Balancer.

 Once you run the above script you will see two files generated as mentioned in the screen below. Copied both file to SOLR_HOME\server\etc folder and then update solr certificate path into the D:\solrcloud\solr\bin\solr.in.cmd batch file.

• Configure Solr configuration

• Increase Java Min/Max Heap size (as per your needs) 

• Update Solr hostname on solr-vm-1, it should solr-vm-2 and solr-vm-3 on other servers

• Update Solr Port and Certificate Path in the Solr.in.cmd file, Make sure the Solr port - 8983 is open at the firewall. You need to install the above generated Solr certificate on -other servers - solr-vm-2, solr-vm-3.

Step 10. Configure ZooKeeper endpoint in the Solr configuration

• Configure the ZooKeeper Ensemble in the Solr.in.cmd file like below, it will same on each Solr server.

• Save Solr.in.cmd file and Start Solr Service and make sure it is running.

• Launch Solr URL - https://solr-vm-1:8983/solr in the browser and verify the things, all three ZK status, graph, etc.

 

Step 11. Create SolrCollection

• Upload Solr ConfigSet - To create the Solr collection we need two configset - Sitecore and xDB Here are the articles to follow for the creation of both configset.

https://doc.sitecore.com/developers/93/platform-administration-and-architecture/en/walkthrough--setting-up-solrcloud.html

https://doc.sitecore.com/developers/93/platform-administration-and-architecture/en/walkthrough--using-solrcloud-for-xconnect-search.html 

 

If we have to make any changes to the Existing configset already uploaded to ZooKeeper we can download the configset and upload it again.

Command to upload configset - 

.\solr zk upconfig -d D:\solr-8.1.1\solr-8.1.1\server\solr\configsets\sitecore_configs -n sitecore1 -z localhost:9986

Command to download configset - 

.\solr zk downconfig -d D:\solr-8.1.1\solr-8.1.1\server\solr\configsets\sitecore_configs -n sitecore1 -z localhost:9986

• Create Collection - To create the collection we can follow the above link to create it via Solr UI Or can use the Collection API to do that. 

$shards=1

         $solrCollectionName= "sitecore_core_index" 

	$replicas=3
	$shardsPerNode=1
	$solrCollectionConfigSet="Sitecore"
	$url = "https://localhost:8983/solr/admin/collections?action=CREATE&name=$solrCollectionName&numShards=$shards&replicationFactor=$replicas&maxShardsPerNode=$shardsPerNode&collection.configName=$solrCollectionConfigSet"
	Invoke-WebRequest -UseBasicParsing -Uri $url | Out-Null

using the above Powershell script we can create the following Sitecore and XDB collections.

• Sitecore Cores -

• "$($collectionPrefix)_core_index",
• "$($collectionPrefix)_master_index",
• "$($collectionPrefix)_web_index"
• "$($collectionPrefix)_marketingdefinitions_master",
• "$($collectionPrefix)_marketingdefinitions_web",
• "$($collectionPrefix)_marketing_asset_index_master",
• "$($collectionPrefix)_marketing_asset_index_web",
• "$($collectionPrefix)_testing_index",
• "$($collectionPrefix)_suggested_test_index",
• "$($collectionPrefix)_fxm_master_index",
• "$($collectionPrefix)_fxm_web_index",
• "$($collectionPrefix)_personalization_index"

• xDBCores -

• "$($collectionPrefix)_xdb",
• "$($collectionPrefix)_xdb_rebuild"

xDB requires to have an Alias of collection, follow links mentioned above to create it from UI.

Once all collections are created, you should see those on the Solr UI.

All Done!

Conclusion:

I hope you have learned something about setting up SolrCoud and ZooKeeper.

Happy Learning! 😊