Before scaling Sitecore Azure Web App, you must read the experience I had and going to share in this post.
Azure Web App
In Azure, you can easily change existent resources almost in real time to accomodate business needs, and this is also applies to Web Apps where you can Scale Up and/or Scale Out your application.
Sitecore
Sitecore offers different architecture models to fit client’s needs, and its roles vary depending on the topology is choosen. Nevertheless, you have the ability to scale either vertical and/or horizontally.
The environment
Everything is hosted in Azure, using Web Apps, and provisioned following Sitecore XP Scaled topology. Here is an idea in how the environment looked like
![sitecore architecture azure paas blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/08/sitecore-architecture-azure-paas-blog-vinicius-deschamps.png)
Before moving forward it is important to know about the difference between Azure Web App and App Service Plans
Azure Web Apps vs App Service Plans
During Azure Web App creation, you are prompted to provide the Instance Details (1) that corresponds to the Web App, and App Service Plan (2) which must be one of the following: create new or choose an existent one from the dropdown list.
![azure azure app service vs azure app service plan create blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/08/azure-azure-app-service-vs-azure-app-service-plan-create-blog-vinicius-deschamps.png)
The relation between Web App and App Service Plan can be illustrated as the image below
![azure app service vs app service plan blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/08/azure-app-service-vs-app-service-plan-blog-vinicius-deschamps.png)
N Web Apps can use the same App Service Plan, as long as the resources the App Service Plan can handle.
In addition to that, an idea that helps to understand is to think like
- App Service Plan is the physical or virtual server which has the resources (CPU, memory, etc)
- Web App is the web server where configuration and settings sits
That said, we wanted to Scale Out both CM and CD, and everything went well with the CD, so let’s attempt to perform the same on CM.
Scale Out CM idea
When you Scale Out your application, a Load Balancer is added in front of your Azure Web App, and Sitecore CM expects exactly that for a horizontal scaling
You can set up multiple instances behind a load balancer. All instances must connect to a centrally located master database
https://doc.sitecore.com/developers/100/platform-administration-and-architecture/en/scaling-and-configuring-content-management.html
Thus we have decided to simply Scale Out the CM Web App
Everything went wrong #1
Once all instances were up and running, a couple tests were made and things were working just fine.
However, when multiple users started to work, things started to become out of control with instances randomically restarting out of the blue with the following message
WARN Shutdown message: CONFIG change
Always look for the tiny details
By scaling out the CM, the environment was going from 1 to N servers, and Sitecore has different requirements to make this to happen
InstanceName parameter
You must ensure that each CM instance has a unique name, and that’s OK because the default value set in Sitecore.config file is (machine name and IIS site name), and when you scale out you will get a clone App Service that runs with a unique name.
<!-- INSTANCE NAME
Unique name for Sitecore instance.
Default value: (machine name and IIS site name)
-->
<setting name="InstanceName" value="" />
Publishing
Only one CM instance can perform publishing, and you must ensure that each CM instance points to the publishing CM instance, and again that’s OK because the environment has a dedicated Publishing Service.
ValidationKey and DecryptionKey
Ensure that the values is the same on all the Sitecore instances within the environment and that the IsolateApps modifier is not present in either value, and in the environment configuration these parameters were set fixed and statically.
Indexing
Since a Scale Out simply clone the file system, and doesn’t allow us to have control in each new server’s file system to modify the Indexing configuration, Sitecore CM was getting confuse with multiple servers performing the indexing role.
Now, that the issue has been identified, the idea was to provisioning a new Content Management using its own App Service (and App Service Plan) where the file system could be modified as needed.
So, let’s call this new Content Management CM02 and the existent CM01.
Enable Indexing sub-role CM02
The idea is to have CM02 with the Indexing sub-role, and to accomplish that a modification must be done in the Web.config
<add key="role:define" value="ContentManagement, Indexing, Indexer"/>
Starting in Sitecore 9.2, when you assign the indexing sub-role to a server role, that role manages content indexing automatically and the other server roles that have not been assisgned this sub-role do not index content
https://dev.sitecore.net/Downloads/Sitecore%20Experience%20Platform/92/Sitecore%20Experience%20Platform%2092%20Initial%20Release/Release%20Notes
Disable Indexing sub-role CM01
IMPORTANT: If you are using Sitecore 9.2 and newer, you don’t need to disable indexing on other CMs
For each Index you have configured, you must change its indexUpdateStrategies to manual like the example below
<index id="sitecore_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex,
Sitecore.ContentSearch.SolrProvider">
<param desc="name">$(id)</param>
<param desc="core">$(id)</param>
<param desc="propertyStore"
ref="contentSearch/indexConfigurations/databasePropertyStore"
param1="$(id)" />
<strategies hint="list:AddStrategy">
<strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/manual" />
</strategies>
Once these changes were applied, here is how the environment look like
![sitecore architecture azure paas scaled out blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitecore-architecture-azure-paas-scaled-out-blog-vinicius-deschamps.png)
Everything went wrong #2
Now, we have everything that is required to have a multiple CM environment
- Load Balancer in front of the CMs
- InstanceName parameter with Unique Names
- A Publishing Server
- Same ValidationKey and DecryptionKey for all CMs
- One CM to handle Indexing
And things were looking very promising, well… well… until the instances from the scaled out CM01 started to fall apart and, again, randomly and with the exactly same message.
WARN Shutdown message: CONFIG change
What else could be wrong? After some research, I found something answered our question
Accordingly to Sitecore Managed Cloud documentation
Scale out is not supported for CM
https://doc.sitecore.com/developers/managed-cloud/en/scaling-options-for-managed-cloud.html
The solution
That said, the only possibility would be to have additional CMs each of them in their own App Service Plan.
However, since we are not scaling the CM App Service anymore, we don’t have a load balancer to split the load between the multiple CMs and there are some important things to know
- Azure Load Balancer does not work with Azure App Services, even if you configure VNet for them
- WAF and Application Gateway are not compatible with Content Management servers
So, you either come up with a third party solution that will assist you in have a Load Balancer for your CMs or you can use Azure Front Door but please keep in mind the following
Use of this technology within a Sitecore solution is permitted, but the Sitecore Support team can only assist with Sitecore product issues that are not unique to the use of the technology, that is, a Sitecore product issue that can be replicated without the technology present.
In some cases, the Sitecore Support team may ask for replication of the issue without the use of the technology, although they will try to avoid this.
https://kb.sitecore.net/articles/682999#Notes
In my case, I am going to use it and here is how it suppose to be
![sitecore content management azure front door blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitecore-content-management-azure-front-door-blog-vinicius-deschamps.png)
Configure Azure Front Door
Search for Front Doors, and click Front Doors in Azure Search field
![azure search front doors blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-search-front-doors-blog-vinicius-deschamps.png)
Click Add to create a new Front Door
![azure front doors add blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-doors-add-blog-vinicius-deschamps.png)
In Basics choose the appropriate Resource Group or create a new one, and Next : Configuration >
![azure create a front door basics blog vinicius deschamps 1](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-basics-blog-vinicius-deschamps-1.png)
At Configuration you will see 3 steps, let’s start by clicking at the + sign on Step 1
![azure create a front door configuration frontends domains blog vinicius deschamps 1](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-configuration-frontends-domains-blog-vinicius-deschamps-1.png)
In Add a frontend host provide Host name (1), ensure to choose Enabled for Session Affinity (2) and disabled for Web Application Firewall (3) – since Sitecore does not like it – and click Add
![azure create a front door configuration frontends domains add a frontend host blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-configuration-frontends-domains-add-a-frontend-host-blog-vinicius-deschamps.png)
Let’s move to the Step 2 by clicking in the + sign
![azure create a front door configuration backend pools blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-configuration-backend-pools-blog-vinicius-deschamps.png)
Add a backend pool by giving it a Name (1) and + Add a backend (2)
![azure create a front door add a backend pool blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-add-a-backend-pool-blog-vinicius-deschamps.png)
Choose a Backend host type (1), in our case an App Service, then choose the Backend host name (2) and its Backend host header (3), and hit Add
![azure create a front door configuration backend pools add a backend app service blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-configuration-backend-pools-add-a-backend-app-service-blog-vinicius-deschamps.png)
Ensure to repeat the step above by clicking + Add a backend until you have all CM Web Apps added
![azure create a front door add a backend pool repeat add a backend app services blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-add-a-backend-pool-repeat-add-a-backend-app-services-blog-vinicius-deschamps.png)
![azure create a front door add a backend pool repeat complete add a backend app services blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-add-a-backend-pool-repeat-complete-add-a-backend-app-services-blog-vinicius-deschamps.png)
Once you have all Web Apps, you can continue on Add a backend pool options, and simply click Add
![azure create a front door configuration backend pools add a backend health probe load balancing blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-configuration-backend-pools-add-a-backend-health-probe-load-balancing-blog-vinicius-deschamps.png)
Finally, let’s configure the Step3 by clicking at + sign
![azure create a front door routing rules blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-routing-rules-blog-vinicius-deschamps.png)
Give it a Name (1), and choose its Frontends/domains (2) and the Backend pool (3), then press Add (4)
![azure create a front door routing rules add a rule blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-routing-rules-add-a-rule-blog-vinicius-deschamps.png)
Now that you have all 3 steps configured hit Review + Create
![azure create a front door review create blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-review-create-blog-vinicius-deschamps-1024x545.png)
Review the information and hit Create
![azure create a front door review create create blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-create-a-front-door-review-create-create-blog-vinicius-deschamps.png)
You should wait until Front Door is provisioned, and you can access from your browser the host name you have set, in my case is https://blog-vinicius-deschamps.azurefd.net
![azure front door access sitecore cm blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-access-sitecore-cm-blog-vinicius-deschamps.png)
Great but you probably don’t want to share an azurefd.net domain with your Content Editors so they can access your CM. Let’s see how to configure a custom domain.
Custom domain Azure Front Door
Go to the Front Door (1), select Front Door designer (2), then click in the + (3) at Frontends/domains
![azure front door front door design frontends domains add new domain blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-front-door-design-frontends-domains-add-new-domain-blog-vinicius-deschamps-1024x177.png)
You should choose a Custom host name, and you will receive a message saying that a CNAME configuration must be done at your DNS
![azure front door add a custom domain blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-add-a-custom-domain-blog-vinicius-deschamps.png)
I am using Azure DNS, so I went to my domain viniciusdeschamps.cloud (1), selected Overview (2) and + Record set (3)
![azure dns new record set blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-dns-new-record-set-blog-vinicius-deschamps-1024x224.png)
In Add record set, add a Name (1), select the Type (2) and add the Alias (3), then hit Ok (4)
![azure dns add record set azure front door cname blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-dns-add-record-set-azure-front-door-cname-blog-vinicius-deschamps.png)
Back to Front Door, and type your Custom host name (1) again, choose Enabled (2) for Custom Domain HTTPS, and in a Production environment use your own certificate but in my case I’m going to use Front Door managed (3), and don’t forget to Enabled (4) Session Affinity and press Add (5)
![azure front door add a custom domain after adding dns cname blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-add-a-custom-domain-after-adding-dns-cname-blog-vinicius-deschamps.png)
In Front Door Designer you will see the following warning
The host ‘your-custom-host.domain’ is not present in any of the routing rules.
The host ‘your-custom-host.domain’ is missing a default route ‘/*’ path for both the HTTP and HTTPS protocolos
![azure front door the host is not present in any of the routing rules blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-the-host-is-not-present-in-any-of-the-routing-rules-blog-vinicius-deschamps.png)
Select the existing Routing rules
![azure front door routing rules update existing rule blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-routing-rules-update-existing-rule-blog-vinicius-deschamps.png)
And in Frontends/domains choose the recently added domain from the list
![azure front door update routing rule frontends domains blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-update-routing-rule-frontends-domains-blog-vinicius-deschamps.png)
Then, press Update
![azure front door update routing rule hit update button after select new frontends domains blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-update-routing-rule-hit-update-button-after-select-new-frontends-domains-blog-vinicius-deschamps.png)
Once you have done, click Save in Front Door designer
![azure front door there are pending changes click save to apply blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-there-are-pending-changes-click-save-to-apply-blog-vinicius-deschamps.png)
![azure front door with custom domain and new rules blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-with-custom-domain-and-new-rules-blog-vinicius-deschamps-1024x195.png)
Please wait for a while until you get a confirmation the configuration has been applied, in my case, it took a while because I choose the Front Door managed certificate
![azure front door updating the custom https configuration blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-updating-the-custom-https-configuration-blog-vinicius-deschamps.png)
![azure front door updated the custom https configuration blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-front-door-updated-the-custom-https-configuration-blog-vinicius-deschamps.png)
Sitecore Identity Server Unauthorized_Client
Once the Azure Front Door was updated with the new Custom Domain, I tried to access https://prodcm.viniciusdeschamps.cloud/sitecore and when hit the Identity Server got the following message
Sorry, there was an error: unauthorized_client
And this happens because in the Identity Server configuration you should set from which URLs authentication requests should be accepted
![sitecore identity server sorry there was an error unauthorized client blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitecore-identity-server-sorry-there-was-an-error-unauthorized_client-blog-vinicius-deschamps.png)
In order to fix it, you should go to Sitecore Identity Server App Service (1), Advanced Tools (2) and Go (3)
![azure app service sitecore identity server advanced tools go blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-sitecore-identity-server-advanced-tools-go-blog-vinicius-deschamps.png)
Select Debug console, and Powershell
![azure app service kudu debug console powershell blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-kudu-debug-console-powershell-blog-vinicius-deschamps.png)
Navigate to site\wwwroot\Config\production, and access the following file Sitecore.IdentityServer.Host.xml then modify the following line
<AllowedCorsOriginsGroup1>https://sc91vdsch-cm.azurewebsites.net|https://</AllowedCorsOriginsGroup1>
![sitecore identityserver host xml add new custom domain blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitecore-identityserver-host-xml-add-new-custom-domain-blog-vinicius-deschamps-1024x573.png)
By adding all Content Management URLs as follows, and hit Save
<AllowedCorsOriginsGroup1>https://sc91vdsch-cm.azurewebsites.net|https://sc91vdsch-cm02.azurewebsites.net|https://sc91vdsch-cm03.azurewebsites.net|https://prodcm.viniciusdeschamps.cloud</AllowedCorsOriginsGroup1>
![sitecore identityserver host xml add new custom domain save blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitecore-identityserver-host-xml-add-new-custom-domain-save-blog-vinicius-deschamps-1024x342.png)
Once the modification is saved, you should Restart the Sitecore Identity Server App Service
![azure app service sitecore identity server overview restart blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-sitecore-identity-server-overview-restart-blog-vinicius-deschamps.png)
![azure app service sitecore identity server are you sure you want to restart blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-sitecore-identity-server-are-you-sure-you-want-to-restart-blog-vinicius-deschamps.png)
Once the restart is completed, try to access Sitecore CM using the custom domain configured on Azure Front Door, in my case https://prodcm.viniciusdeschamps.cloud/sitecore and voilá no error
![sitcore identity server fully accessible type credentials blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitcore-identity-server-fully-accessible-type-credentials-blog-vinicius-deschamps.png)
I got redirect to the Sitecore Experience Platform, and as you can see to the CM02 but that’s not I want for my end users, so there are additional steps to address it. Well, at least we know the Front Door is splitting the traffic
![sitecore not using azure front door custom domain blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitecore-not-using-azure-front-door-custom-domain-blog-vinicius-deschamps.png)
Custom Domains for App Services
Navigate to the App Services, and choose one of the CMs (1), select Custom Domains (2) and copy the Custom Domain Verification ID (3)
![azure app service custom domains custom domain verification id blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-custom-domains-custom-domain-verification-id-blog-vinicius-deschamps-1024x539.png)
Go to your DNS provider, in my case Azure DNS, and let’s add a new Record Set
![azure dns new record set blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-dns-new-record-set-blog-vinicius-deschamps-1024x224.png)
The step below is to check domain ownership, and you should fill the information as follows
- Name – asuid.SUBDOMAIN (e.g asuid.prodcm)
- Type – TXT
- Value – Paste the value you copied from Custom Domain Verification ID
![azure dns add record set custom domain verification txt blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-dns-add-record-set-custom-domain-verification-txt-blog-vinicius-deschamps.png)
Back to the Custom Domains, and click Add custom domain (3)
![azure app service custom domains add custom domain blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-custom-domains-add-custom-domain-blog-vinicius-deschamps-1024x547.png)
Type your Custom Domain (1), then click Validate (2)
![azure app service add custom domain validate blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-add-custom-domain-validate-blog-vinicius-deschamps.png)
Sometimes it take a while until the record is sync, and you might see the following message
Domain ownership
To verify domain ownership create CNAME and TXT record with your DNS provider
Since we already have a CNAME configured for the Azure Front Door, the CNAME record is not required
![azure app service add custom domain add custom domain domain ownership blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-add-custom-domain-add-custom-domain-domain-ownership-blog-vinicius-deschamps.png)
Wait for a while and try to Validate the Custom Domain again, and if everything is OK, you should see the following and hit Add custom domain
![azure app service add custom domain add custom domain blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/azure-app-service-add-custom-domain-add-custom-domain-blog-vinicius-deschamps.png)
IMPORTANT: You should add the same custom domain to every CM App Service!
And finally, when you access your Azure Front Door custom domain to hit Sitecore you will notice that now you have a single URL for all Content Management, which in my case is https://prodcm.viniciusdeschamps.cloud
![sitecore using azure front door custom domain blog vinicius deschamps](https://viniciusdeschamps.com.br/wp-content/uploads/2020/09/sitecore-using-azure-front-door-custom-domain-blog-vinicius-deschamps.png)
And that’s it!
I hope you liked, and I’ll see you on my next post.
Photo by Marius Masalar on Unsplash
Deixe um comentário