Sitecore Marketing Automation Engine not starting

25/11/2022 / Parag Daraji / Leave a comment

When investigating on dynamic environment setup and configuration on Azure. I came across an issue when Marketing Automation Engine worker process was going back to Pending restart state.

The Issue

The Sitecore Marketing Automation engine is not starting up. In spite of manually start again it going back to PendingStart state.

The Solution

First thing is to check the logs. When I see the logs for marketing automation engine webjob. It was having below given error message.

[11/24/2022 05:20:10 > 701b8b: SYS INFO] Run script 'Sitecore.MAEngine.exe' with script host - 'WindowsScriptHost'
[11/24/2022 05:20:11 > 701b8b: SYS INFO] Status changed to Running
[11/24/2022 05:20:11 > 701b8b: INFO] 
[11/24/2022 05:20:11 > 701b8b: INFO] Starting Marketing Automation Engine...
[11/24/2022 05:20:12 > 701b8b: ERR ] ERROR
[11/24/2022 05:20:12 > 701b8b: ERR ] System.Xml.XmlException: Data at the root level is invalid. Line 1, position 1.
[11/24/2022 05:20:12 > 701b8b: INFO] 
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlTextReaderImpl.Throw(Exception e)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlDocument.Load(XmlReader reader)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlDocument.LoadXml(String xml)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at ?????????????????????????????????????????.?????????????????????????????????????????(????????????????????????????????????????? , String )
[11/24/2022 05:20:12 > 701b8b: ERR ]    at ?????????????????????????????????????????..ctor(String )
[11/24/2022 05:20:12 > 701b8b: ERR ]    at ?????????????????????????????????????????..ctor(String )
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.XConnect.Configuration.Extensions.InitializeLicenseCheck(IServiceCollection collection, String licenseFileOrXml)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.XConnect.Configuration.Extensions.UseXConnectServiceInitializationConfiguration(IServiceCollection collection, IConfiguration configuration, String[] configurationSectionNames, String initializationSectionName, Boolean validateConfiguration)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.Xdb.MarketingAutomation.Engine.App.ServiceConfiguration.ConfigureServices(IConfiguration configuration, Boolean useDecorators)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.Xdb.MarketingAutomation.Engine.App.ServiceConfiguration.Reload()
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.Xdb.MarketingAutomation.Engine.App.Program.RunAsConsoleApp(Boolean interactive)
[11/24/2022 05:20:12 > 701b8b: SYS INFO] Status changed to Success
[11/24/2022 05:20:12 > 701b8b: SYS INFO] Process went down, waiting for 60 seconds
[11/24/2022 05:20:12 > 701b8b: SYS INFO] Status changed to PendingRestart

At a first glance, above error message does not provide any valuable clue. But when read it fully and after few reads, found out the issue is related to license file. The InitializeLicenseCheck method indicate it clearly. Checked the license file on the MA engine job directory and the file contains only the text “LicenseContent” which is obviously not a valid xml format and there is no root element…!!!

In this Marketing automation Operations web app, there are two license files. One for the web application under the path <website root directory>\App_Data\license.xml and another one for MA engine worker job for which the path of license file is <website root directory>\App_Data\jobs\continuous\AutomationEngine\App_Data\license.xml.

Replace the content of both the license files with the correct license xml content. Stop and start the webjob and web application and the happy days…!!!!

Posting this message as I could not found any blog providing any solution about this error message. Hope this helps my fellow Sitecorians!

Configure .Net Remote Debugging Service in Sitecore 10.2 Docker

05/05/202206/05/2022 / Parag Daraji / Leave a comment

Hi Sitecorians,

It’s been very long since, I last blog. But the wait is over and I am here with some of my learning while upgrading our Siteore platform from 9.3 to 10.2( latest and greatest). With this project, there are lots of learning and I’ll try to share it with you community.

The Issue

After deploying the customisation on top of the vanilla Sitecore 10.2 (huge applaud to Developers of Sitecore and the vibrant community to create and maintain the Docker sample repository), the first obvious step is to do a quick sanity test to check whether or not anything is broken??…. and the unexpected happens, one of the custom written pipeline code was complaining and logging errors in the CM log file.

So the next best handy tool for a developer in Dot Net world is the .Net debugger( thank God). While, I was trying to attache to w3wp process of CM container the following error appeared.

Unable to connect to the Microsoft Visual Studio Remote Debugger named 'XXX.XXX.XXX.XXX'.  The Visual Studio 2022 Remote Debugger (MSVSMON.EXE) does not appear to be running on the remote computer. This may be because a firewall is preventing communication to the remote computer. Please see Help for assistance on configuring remote debugging.

The Solution

As per the error message above, it’s clear, there is something not right with the MSVSMON.EXE.

Immediately fire up following PowerShell commands to look into running processes under the CM container.

PS> Docker exec -it <<CM Container name/id>> powershell
PS> Get-Process

The output of the Get-Process command was not showing any currently running process with name msvsmon. uuhhhh… that is the issue, I guess…!!!

So to get the .Net remote debugging working on the CM container, I did checked how we had it working in our 9.3 sitecore Docker Environment. Found 3 different steps to get it working. I wanted to document is in my diary but did not got chance. It is never to late.

Lets have a look at those 3 steps one after another.

Step 1.

Open the docker-compose.override.yml file, add below line under the volumes

 cm:
    image: ${REGISTRY}${COMPOSE_PROJECT_NAME}-xp0-cm:${VERSION:-latest}
    build:
      context: ./docker/build/cm
      args:
        BASE_IMAGE: ${SITECORE_DOCKER_REGISTRY}sitecore-xp0-cm:${SITECORE_VERSION}
        SPE_IMAGE: ${SITECORE_MODULE_REGISTRY}sitecore-spe-assets:${SPE_VERSION}
        SXA_IMAGE: ${SITECORE_MODULE_REGISTRY}sitecore-sxa-xp1-assets:${SXA_VERSION}
        TOOLING_IMAGE: ${SITECORE_TOOLS_REGISTRY}sitecore-docker-tools-assets:${TOOLS_VERSION}
    volumes:
     - ${REMOTEDEBUGGER_PATH}:C:\remote_debugger:ro

Step 2.

Next step to add the REMOTEDEBUGGER_PATH variable to the .env file. To do so open your .env file in VS Code or your choice of text editor and add below line to it.

REMOTEDEBUGGER_PATH=C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\IDE\Remote Debugger\x64

Now, you need to change the path of debugger folder in above with the VS version you are using. For, now I am using VS 2022 Professional but we have some developers still using VS 2019 Professional for them the path will be C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Remote Debugger\x64

Step 3.

Now, in the third and the final step, we will try to execute multiple processes at the entry point for CM container. To do so, I have added few commands to the existing Powershell entry point command.

 cm:
    image: ${REGISTRY}${COMPOSE_PROJECT_NAME}-xp0-cm:${VERSION:-latest}
    build:
      context: ./docker/build/cm
      args:
        BASE_IMAGE: ${SITECORE_DOCKER_REGISTRY}sitecore-xp0-cm:${SITECORE_VERSION}
        SPE_IMAGE: ${SITECORE_MODULE_REGISTRY}sitecore-spe-assets:${SPE_VERSION}
        SXA_IMAGE: ${SITECORE_MODULE_REGISTRY}sitecore-sxa-xp1-assets:${SXA_VERSION}
        TOOLING_IMAGE: ${SITECORE_TOOLS_REGISTRY}sitecore-docker-tools-assets:${TOOLS_VERSION}
    volumes:
      - ${REMOTEDEBUGGER_PATH}:C:\remote_debugger:ro
    networks:
      dev.local:
        aliases:
          - cm.dev.local
    entrypoint: powershell -Command "& Write-Host 'Starting Debugger Service...'; & Start-Process -FilePath 'C:\remote_debugger\msvsmon.exe' -ArgumentList '/nostatus', '/silent', '/noauth', '/anyuser', '/nosecuritywarn', '/wow64port 4026'; & C:\tools\entrypoints\iis\Development.ps1"

With the above alteration firstly, I am writing to host to get to know that it is trying to start debugger service and secondly executing MSVSMON.EXE file with few arguments as Start-Process powershell command.

With the argument wow64port mind that each Visual Studios have their own ports assigned as default debugging port for msvsmon.exe. See the references in section below.

Verification

As verification of both w3wp and msvsmon services running we need to check the output of Get-Process command on CM container

Note
Also make sure in the web.config of your CM container, the debug is set to true for compilation node. configuration/system.web/compilation[debug=’true’]

Note
Notice that I am using (;) semicolon as the command terminator for powershell command. If you are using CMD, use & (ampersand)

References

https://docs.microsoft.com/en-us/visualstudio/debugger/remote-debugger-port-assignments?view=vs-2022

https://ss64.com/ps/call.html

https://github.com/microsoft/DockerTools/issues/91

Event Queue Trouble shooting

18/10/2021 / Parag Daraji / 2 Comments

Recently we have got an weird issue reported by our content authors on production. They mentioned the content are not reflected on website after they been publish or even re-publish.

First I wanted to check does the Publishing service doing all right and publishing contents correctly. Checked and it was working absolutely fine. I did switch to web database and validated recently published content. After some initial trouble shoot, discovered on CDs, the cache is not been clearing upon successful publish.

This leads my thought to suspect event queue table may have been flooded with lots of entry and may need a clean.

Did a eventqueue table cleaning as mentioned here and things did not went to normal. Still published changes were not reflecting on the website.

Now, I need to dive into detail to understand what is happening in eventqueue table.

The Properties table holds the last event execution time stamp against all the CDs.

To get that last run timestamp follow below steps:

go to your web app and open the Kudu advance tool.
Navigate to Environment page from menu in header
Get the Machine Name from System Info section

Open SSMS and connect to Database
Open new query window for the database which is configured as your Event Queue database( in our case we have configured dedicated one. By default it is web)
Run below query to get the last run time stamp for that CD instance

SELECT * FROM Properties where Key LIKE '%WEB_EQSTAMP_<<MACHINENAME>>%' ORDER BY Value DESC

Replace<<MACHINENAME>> in above query with machine name from step 3 above

Depending on how may instances your selected CD is running, it will show you that may rows (by default there will be two rows for each CD. one is for Production instance and another for pre-prod instance). The value column holds the Time stamp value of last run.

Now, that we have the last run time stamp for the CD, we can get the list of all pending events for this CD instance by running below query.

  SELECT * FROM EventQueue WHERE Stamp >= CONVERT(VARBINARY, <<TIMESTAMP>>) ORDER BY Created DESC

Replace the <<TIMESTAMP>> with the value from previous query.

By combining all queries together, the final query will look like below

SELECT * FROM EventQueue WHERE Stamp >= 
  (
    SELECT CONVERT(VARBINARY, Value) from [Properties] where [Key] = 'WEB_EQSTAMP_<<MACHINENAME>>'
  )
  ORDER BY [Created] DESC

After all this investigation, it is clear that CD servers are not triggering remote events registered in dedicated event queue table.

Did little more investigation and found out wrong event queue configuration files has been deployed to CDs accidentally and that was causing this behavior. We placed the correct config files for dedicated event queue and the peace has been established on CD role again.

Happy Event queue trouble shooting…!!!

References

Sitecore Docker – Run CM sites on HTTPS

22/03/202121/08/2021 / Parag Daraji / Leave a comment

While working on the integrating Sitecore’s CM’s functionality with third party Digital Asset Management(DAM) system, I came across one interesting issue where I wanted my container applications to run on https.

The Issue

After integrating the third party DAM with Sitecore CMS, it found out that, while using local website on developer machine, the developer is not able to load the SSO login page. While loading that page below error messages logged in browser’s console.

Access to the WebCrypto API is restricted to secure origins. Compact View requires HTTPS when used outside localhost (for development).

As clearly stated I am not using localhost as host to access my cm and seems like CM needs to be securely severed over https where as we, for local development, use non secure protocol http.

The Fix

The solution is simple looking form 35,000 feet. Needs CM urls to be serving content over the secure HTTPS protocol. But when actually started digging in details, it is fun and very learning experience.

Performed following steps to achieve this on our local docker development environment.

Clone or download below repository

https://github.com/michaellwest/docker-https

Open the startup/createcert.ps1 for editing and change following parameter’s default value

$certificatepassword – from b to more secured passowrd string
$dnsNameList – by default the value is *.dev.local. Change this value to match your host pattern. You can specify individual host comma(,) separated.

Note
Alternatively, you can can pass those two as parameter while firing createcert.ps1 command.

3. Open docker-compose.yml file and navigate to cm service and perform following changes:

Add new environment parameter HOST_HEADER and set cm host value to it. If you have multiple host(this is what in my case), specify them semicolon(;) separated list. For example host1.dev.local;host2.dev.local
Under the volumes bind the new volume for folder startup to c:\startup path of container.
Under the port bind the https’s default secure port 443 with the next available container port

Note
The container host port needs to be new port which is not been use by any other services.

Lastly, change the entypoint to the path of startup.ps1 file under the startup folder bound under the volumes in above step for startup folder.

After all above changes to CM service of docker-compose.yml file will look something like below

services:
  ...
  cm:
    image: ${REGISTRY}sitecore-xm-cm:${SITECORE_VERSION}-windowsservercore-${WINDOWSSERVERCORE_VERSION}
    entrypoint: powershell.exe -NoLogo -NoProfile -File C:\\startup\\startup.ps1
    volumes:
      - .\src:C:\src
      - .\startup:C:\startup
    ports:
      - "44001:80"
      - "44002:443"
    networks:
      default:
        aliases:
          - cm.dev.local
          ...
    environment:
      HOST_HEADER: cm.dev.local
      ...

4. Copy folder startup and it’s content to same folder where your docker-compose.yml file is.

5. Open the powershell in elevated mode and execute following command

PS> cd <<docker-compose.yml file's folder path>>
PS> ./startup/createcert.ps1

Above powershell script will generate three files cert.cer, cert.pfx and cert.password.text under the startup folder.

This command will also generate the self-signed wildcard certificate and install to your system’s local certificate store under personal.

To verify that run mmc and open local computer certificate store.

Self signed wildcard certificate for *.dev.local

6. Now, is the time to fire up the container for sitecore. To do so run following docker compose command

PS> docker-compose up -d

After successful execution the container services will be up and one can access the host binding with https. For example https://cm.dev.local in this case.

Advance trouble shooting

To check the certificate is exist after the createcert.ps1 script execution. This command can be run on both your local and against cm container as well.

PS> Get-ChildItem -Path cert:\LocalMachine\Root


PS> Get-ChildItem -Path cert:\CurrentUser\Root

To verify the binding under the container’s IIS for the specified hosts use following commands

PS> Get-Website -Name 'Default Web Site'

To get all the bindins and their port details for a website

PS> Get-WebBinding -Name 'Default Web Site'

References

Sitecore Publishing Service – Part 1 – Install and Configure Service in Azure PaaS

06/06/201928/03/2020 / Parag Daraji / Leave a comment

While working on finding solution for slow publishing issue of more than 10,000 items in Azure PaaS, I came across this very powerful add-on Sitecore Publishing Services.

Believe it or not but once it is installed, configured and up, it reduced the publishing time drastically. In our case, publishing of those 10,000 items were taking around 1 hour 30 mins(hhhuuuffff…), merely took 12 mins to publish.(Super duper awesome, isn’t it…??? This makes everyone’s life easy)

Thus, I decided to write blog on whole solution. Blog will become very lengthy if I put everything in one. Thus, splitting it in three parts.

Part 1 will cover installation and basic configuration of publishing service. Part 2 will cover installation of publishing module in CMS. The third part will cover advance configuration of publishing service.

So, lets get started…

Download

You can download both Sitecore publishing service and publishing module from Sitecore’s development site here. It is the designed and developed by Sitecore itself.

Note:
Just check the version compatibility with Sitecore version where you wanted to implement this publishing service/module. see image below for more details. (As I have downloaded first the latest and greatest version 4 at the time I am writing this blog for Sitecore 9.0.1 and later figure out it is not compatible.)

Install publishing service on Azure PaaS

With my current client all their Sitecore implementation is in Azure PaaS. It absolutely makes sense to have this Sitecore Publishing service also setting in Azure PaaS. So, follow below steps to get the publishing services up as app service.

Download the service application as zip file from above link. Select the one which is matching your host architecture type either 32 bit or 64 bit.
Open Kudu – Advance tool for the web app where you want to host publishing service

in Kudu editor open command console by navigate to Debug Console menu option and select CMD
navigate to site root folder using following command

cd site\wwwroot

Drag and drop service zip file to folder view area above console. This will upload the service package to Azure web app.
Once upload completed, run following command on console below the folder view

unzip -o *.zip

Assumption
The site’s wwwroot folder has only one zip file which is recently uploaded publishing service zip file.

Once unzipping of application completed, execute following commands in CMD. This will set connection stings of core, master and web database in publishing service. Going forward, these connection strings will be used for all publishing jobs.

Sitecore.Framework.Publishing.Host configuration setconnectionstring core "<core database connection string>"

Sitecore.Framework.Publishing.Host configuration setconnectionstring master "<master database connection string>"

Sitecore.Framework.Publishing.Host configuration setconnectionstring web "<web database connection string>"

Once connection strings are configured for these databases the next step is to upgrade the database schema to required one for Sitecore publishing service

Sitecore.Framework.Publishing.Host schema upgrade --force

After executing above command the console displays the result of the command. The below screenshot have result with already updates schema for all the databases as I have previously upgraded the schema.

Once above commands are successfully executed, now it is the time to check the status of the publishing service web application.

In the browser open below URL

http://<<azure app service name>>.azurewebsites.net/api/publishing/maintenance/status

Above web api will return following JSON result if all the configurations are correctly set.

{
    "status": 0
}

This are basic steps to deploy Sitecore Publishing services up and running in azure PaaS. In the Part – 2, will cover installation of Sitecore Publishing Module in Sitecore CM/CA with configuration.

Content sync with RAZL script

22/03/201725/06/2019 / Parag Daraji / Leave a comment

Hi Sitecore Enthusiastic,

Recently come across very great strategy to sync content between two instances of Sitecore using a very powerful tool called Razl form Hedgehog.

Please read more about Razl here.

I have used this tool before mainly to compare the content between two different databases. For example production CM and Pre-production CM. Today, I came across very new and great features with latest version of Razl offering call tasks and executing task using Razl scripts.

It is offering content sync tasks to be exported in a file(or call script) as xml format and later one can use that to sync data in different environment using Razal script.

For more information about Razl task, read official document here.

The great part of this task is you can cherry pick what you want to sync out of the whole Sitecore content tree. You can go deep up to the field level with your sync task.

once you have a list of tasks ready, you simple can export that as script and execute it with command or create scheduler task to run that script using command and it will do the sync between source and target connections.

Razl.exe /script:Migrate.xml
Razl.exe /script:"c:\Site Migration\Razl.xml"

Even if you want to go one level further you can automate this sync task post publishing(perhaps not ideal, if frequent publishing is happening) or you can hook it post deploy task in you CD script(if your build contains TDS/unicorn content items).

You must be wondering this is great tool and great way to sync, but what if I am trying to sync big chunk of content items!!!???.
You can use another great feature of Razl called Lightening Mode(obviously you can configure this in you script as well). This mode really boost the performance of sync script if your CoplyAll task is configured to do bulk copying of items.

As far as logging concern, Razl Script mode will output actions as they happen to the command prompt, this output can be written to a text file if required using following command sample.

Razl.exe /script:"c:\Site Migration\Razl.xml" >> c:\logs\RazlScriptLogs.txt

Tip: Single arrow > will create the file if missing and if exist, overwrite the content of file.
Double arrow >> will create the file if missing and if exist, it will append to the file.

If a log folder has been already configured in the Razl GUI than Script mode will also write to the log file under configured log folder.

Thanks for reading.

Happy Content Sync…!!!

Reference
Script – https://hedgehogdevelopment.github.io/razl/script.html
Connections – https://hedgehogdevelopment.github.io/razl/connections.html
Lightning mode – https://hedgehogdevelopment.github.io/razl/comparing.html#lightning-mode

Content Editor – Create new version automatically on Edit

18/05/201605/09/2016 / Parag Daraji / Leave a comment

I came across very nice feature Sitecore CMS is providing out-of-box. This might be very useful strategy while doing the content population/editing.

Here, I am trying to configure Content Management(CM) or say Content Authoring(CA) server to automatically create newer version upon editing the item. This whole exercise can be divide in to two steps :

Step 1: Configure Locking is requited for item editing

To do this, you may need to open the web.config and check below setting’s value is set to “true” on CA server. The default value is true.


<setting name="RequireLockBeforeEditing" value="true" />

Step 2: Configure auto-creation of version for item

To get this thing working, default workflow must be assigned to the item for which automatic version creation should be enable while editing. As a start, you can use the default workflow provided by Sitecore /sitecore/system/Workflows/Sample Workflow and if you have required workflow ready you can utilize that. If you planning to utilize sample workflow, please also consider reading my this post.

The best practice is to assign default workflow to template’s standard values. Have a look to below image for more details.

Template Standard Values - Default Workflow

Tip: You may need to switch on the Standard Fields view form View Menu to see Workflow section.

Following code in WorkflowContext class of Sitecore.Workflows.dll is actually responsible for creating version on editing.


public Item StartEditing(Item item)
{
	Error.AssertObject(item, &amp;quot;item&amp;quot;);
	if (!Settings.RequireLockBeforeEditing || Context.User.IsAdministrator)
	{
		return item;
	}
	if (this._context.IsAdministrator)
	{
		return this.Lock(item);
	}
	if (StandardValuesManager.IsStandardValuesHolder(item))
	{
		return this.Lock(item);
	}
	if (!this.HasWorkflow(item) &amp;amp;&amp;amp; !this.HasDefaultWorkflow(item))
	{
		return this.Lock(item);
	}
	if (!this.IsApproved(item))
	{
		return this.Lock(item);
	}
	Item item2 = item.Versions.AddVersion();
	if (item2 != null)
	{
		return this.Lock(item2);
	}
	return null;
}

All above works for non administrator users. By default, for admin user in Sitecore, they do not need to lock the item to edit so it does not triggers any workflow.

Happy Versioning…!!! 🙂

Ongoing publishing issue with Sitecore 7.2

26/04/201611/10/2016 / Parag Daraji / 2 Comments

Here I wanted to share simple solution to an ongoing issue of publishing with Sitecore 7.2 with my company. It is such a simple solution but took lots of efforts and time to get to it. Thought to share it so that if someone is facing same kind of issue, can be benefited out of it.

The problem:

Out marketing team(who is also responsible for the content management) continuously reporting us with the issue of publishing. They mentioned that whenever we try to publish any item, the publishing took lot of time to finish. Most of the time publishing dialog stuck at Initialization stage. This issue sometimes go worst, even after waiting for half an hour to 45 minutes the publishing dialog still shows initialization and we need to ask web admin guys to re-cycle application pool and start publishing again(this is worst case scenario).

So as a first step, I asked them to avoid bulky publishing. I also suggested then to avoid using publishing related items rather go to individual items changed and only publish those items. The reason is to minimize items count in publishing which keeps publishing process as short as possible and keep them moving.

Even though content authors follow my suggestions, the issue still persist. So we started investigating even more. We planned to take help of Sitecore support personal who visiting us once every month. He suggested us to compare all the config files with pure vanilla version and along with all the other files and folder so we can be absolute sure we are having all the goodies what should be there. After this huge activity we identified couple of missing files and configurations in both CM and CD environments.

So we have gathered all those missing pieces and deployed to the respected environments.

After this we have reported a side effect from marketing, that what ever we are deploying to Fail-over publishing target(which is our DR cum internal Preview servers) it also eventually published to production(which is very strange to us and this was not the behavior before we deploy the missing bits).

Root cause

So, I started investigating in context to new side effects. While going through the log file for CM server, my eyes struck with an strange log entry which looks something like below.

5768 2016-03-10 11:03:57 INFO AUDIT (ad\paragd): Execute workflow command. Item: master:/sitecore/content/Site1/Home/test, language: en, version: 11, id: {23005A37-6BB5-4025-AC5B-047A8D3F3719}, command: /sitecore/system/Workflows/Site1 Workflow/Draft/Submit, previous state: Draft, next state: /sitecore/system/Workflows/Site1 Workflow/Awaiting Approval, user: ad\paragd
ManagedPoolThread #3 2016-03-10 11:03:59 INFO Starting update of index for the database 'master' (1 pending).
ManagedPoolThread #3 2016-03-10 11:03:59 INFO Update of index for the database 'master' done.
ManagedPoolThread #16 2016-03-10 11:03:59 INFO Starting update of index for the database 'master' (1 pending).
ManagedPoolThread #16 2016-03-10 11:03:59 INFO Update of index for the database 'master' done.
5284 2016-03-10 11:04:00 INFO AUDIT (ad\paragd): Execute workflow command. Item: master:/sitecore/content/Site1/Home/test, language: en, version: 11, id: {23005A37-6BB5-4025-AC5B-047A8D3F3719}, command: /sitecore/system/Workflows/Site1 Workflow/Awaiting Approval/Approve, previous state: Awaiting Approval, next state: /sitecore/system/Workflows/Site1 Workflow/Approved, user: ad\paragd
5284 2016-03-10 11:04:00 INFO AUDIT (ad\paragd): [Publishing]: Starting to process 2 publishing options
4572 2016-03-10 11:04:00 INFO HttpModule is being initialized
568 2016-03-10 11:04:00 INFO HttpModule is being initialized
5456 2016-03-10 11:04:01 INFO Job started: Publish
2736 2016-03-10 11:04:01 INFO Job started: Publish to 'web'
.
.
.
2736 2016-03-10 11:04:01 INFO Job ended: Publish to 'web' (units processed: 11)
7052 2016-03-10 11:04:01 INFO Job started: Publish to 'web_Failover'
.
.
.
7052 2016-03-10 11:04:02 INFO Job ended: Publish to 'web_Failover' (units processed: 11)

Above lines, I found in the log just above the log line for manually triggering of publishing just for the single publishing target which is “fail-Over”.

4392 2016-03-10 11:04:18 INFO AUDIT (ad\paragd): Publish item: master:/sitecore/content/Site1/Home/test, language: en, version: 11, id: {23005A37-6BB5-4025-AC5B-047A8D3F3719}
3600 2016-03-10 11:04:28 INFO AUDIT (ad\paragd): Publish, root: {23005A37-6BB5-4025-AC5B-047A8D3F3719}, languages:en, targets:WWW_Failover, databases:web_Failover, incremental:false, smart:true, republish:false, children:false, related:false
3600 2016-03-10 11:04:28 INFO AUDIT (ad\paragd): [Publishing]: Starting to process 1 publishing options
2864 2016-03-10 11:04:29 INFO Job started: Publish
5240 2016-03-10 11:04:29 INFO Job started: Publish to 'web_Failover'

Solution

From above behavior on the log file, it was clear that the workflow has to do something with this. When I checked the default workflow assigned, I found that there is an auto publishing action configured with final stage of the workflow as shown in below screen grab.

But alone having auto publish action configured, should not be making publishing this slow. So I have investigated more and found the real culprit.

The auto published action is configured with deep=1 as parameters.
So now, I have dive into the de-compiled code and found this is very dangerous parameter configured with auto publishing action.
Having deep parameter set, you are instructing auto publisher to publish everything underneath the selected item(approving item here) to every publishing targets(in this case 2 targets).

So, if you are approving leaf node/item, the auto publishing will be quick as it will only publish that node along with no child. But if you are approving, lets say home node of the site, you are actually publishing not just that root home item but pretty much the whole site to all available publishing targets…!!!(Very shocking…isn’t it?).

So now after approving the item, when you trying to publish that home node to selected target, it always queue up in the publishing queue and Sitecore publishing dialog box show you Initialization message screen as shown above. The item publishing will be picked up when previous auto publishing job is done. This is the case with just one editor, if you have 5-6 content editor, then the situation can go into really worst.

So the easy solution is to remove that auto publishing action item which was publishing lot of unwanted items to all targets in background without any visible signs.

So we did that and everyone is happy ever after…..

But the story does not ended here. I was thinking while going to home that day, when the auto publishing action was configured and present with default workflow since the edges then why approving the item wasn’t picked by both the CD servers?

So next day, I have kept my investigation going and discover the ScalabilitySettings.config file which was missing on all CD servers. While deploying the missing configurations and file we have deployed this file on all CD serves started picking the events execution for all CDs.

So, happy publishing….!!!!

Reference

https://sitecorebasics.wordpress.com/2011/05/28/is-your-sitecore-publishing-stucks/

Cooking with Sitecore

Diary of my experience working on great CMS called Sitecore

Architecture

Sitecore Marketing Automation Engine not starting

The Issue

The Solution

Configure .Net Remote Debugging Service in Sitecore 10.2 Docker

The Issue

The Solution

Step 1.

Step 2.

Step 3.

Verification

References

Event Queue Trouble shooting

References

Sitecore Docker – Run CM sites on HTTPS

The Issue

The Fix

Advance trouble shooting

References

Sitecore Publishing Service – Part 1 – Install and Configure Service in Azure PaaS

Download

Install publishing service on Azure PaaS

Content sync with RAZL script

Content Editor – Create new version automatically on Edit

Ongoing publishing issue with Sitecore 7.2