Sitecore Marketing Automation Engine not starting

When investigating on dynamic environment setup and configuration on Azure. I came across an issue when Marketing Automation Engine worker process was going back to Pending restart state.

The Issue

The Sitecore Marketing Automation engine is not starting up. In spite of manually start again it going back to PendingStart state.

The Solution

First thing is to check the logs. When I see the logs for marketing automation engine webjob. It was having below given error message.

[11/24/2022 05:20:10 > 701b8b: SYS INFO] Run script 'Sitecore.MAEngine.exe' with script host - 'WindowsScriptHost'
[11/24/2022 05:20:11 > 701b8b: SYS INFO] Status changed to Running
[11/24/2022 05:20:11 > 701b8b: INFO] 
[11/24/2022 05:20:11 > 701b8b: INFO] Starting Marketing Automation Engine...
[11/24/2022 05:20:12 > 701b8b: ERR ] ERROR
[11/24/2022 05:20:12 > 701b8b: ERR ] System.Xml.XmlException: Data at the root level is invalid. Line 1, position 1.
[11/24/2022 05:20:12 > 701b8b: INFO] 
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlTextReaderImpl.Throw(Exception e)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlDocument.Load(XmlReader reader)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at System.Xml.XmlDocument.LoadXml(String xml)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at ?????????????????????????????????????????.?????????????????????????????????????????(????????????????????????????????????????? , String )
[11/24/2022 05:20:12 > 701b8b: ERR ]    at ?????????????????????????????????????????..ctor(String )
[11/24/2022 05:20:12 > 701b8b: ERR ]    at ?????????????????????????????????????????..ctor(String )
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.XConnect.Configuration.Extensions.InitializeLicenseCheck(IServiceCollection collection, String licenseFileOrXml)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.XConnect.Configuration.Extensions.UseXConnectServiceInitializationConfiguration(IServiceCollection collection, IConfiguration configuration, String[] configurationSectionNames, String initializationSectionName, Boolean validateConfiguration)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.Xdb.MarketingAutomation.Engine.App.ServiceConfiguration.ConfigureServices(IConfiguration configuration, Boolean useDecorators)
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.Xdb.MarketingAutomation.Engine.App.ServiceConfiguration.Reload()
[11/24/2022 05:20:12 > 701b8b: ERR ]    at Sitecore.Xdb.MarketingAutomation.Engine.App.Program.RunAsConsoleApp(Boolean interactive)
[11/24/2022 05:20:12 > 701b8b: SYS INFO] Status changed to Success
[11/24/2022 05:20:12 > 701b8b: SYS INFO] Process went down, waiting for 60 seconds
[11/24/2022 05:20:12 > 701b8b: SYS INFO] Status changed to PendingRestart

At a first glance, above error message does not provide any valuable clue. But when read it fully and after few reads, found out the issue is related to license file. The InitializeLicenseCheck method indicate it clearly. Checked the license file on the MA engine job directory and the file contains only the text “LicenseContent” which is obviously not a valid xml format and there is no root element…!!!

In this Marketing automation Operations web app, there are two license files. One for the web application under the path <website root directory>\App_Data\license.xml and another one for MA engine worker job for which the path of license file is <website root directory>\App_Data\jobs\continuous\AutomationEngine\App_Data\license.xml.

Replace the content of both the license files with the correct license xml content. Stop and start the webjob and web application and the happy days…!!!!

Posting this message as I could not found any blog providing any solution about this error message. Hope this helps my fellow Sitecorians!


Sitecore CMS not showing graph on Launch pad

This morning when I login to one of none production environment I found that the graph was not loading on Sitecore CMS launch pad.

When I did into more details and found out the ajax request to get the data are failing with server error(HTTP-500). As shown in below image the response of the API call does not providing any information or clue.

Immediately, I jumped onto logs and found following entry in there.

 The remote server returned an error: (403) Forbidden.

The full exception looks like below.

50616 09:48:50 ERROR [Experience Analytics]: System.Net.WebException: The remote server returned an error: (403) Forbidden.
   at System.Net.HttpWebRequest.GetResponse()
   at Sitecore.Xdb.Reporting.Datasources.Remote.RemoteReportDataSourceProxy.GetData(ReportDataQuery query)
   at Sitecore.Xdb.Reporting.ReportDataProvider.c__DisplayClass18_0.b__1()
   at Sitecore.Xdb.Reporting.ReportDataProvider.GetCachedDataForQuery(ReportDataQuery query, Func`1 ifNotAvailableInCache, Nullable`1 ifModifiedSince)
   at Sitecore.Xdb.Reporting.ReportDataProvider.ExecuteQueryWithCache(ReportDataQuery query, ReportDataSource dataSource, CachingPolicy cachingPolicy)
   at Sitecore.Xdb.Reporting.ReportDataProvider.GetData(String dataSourceName, ReportDataQuery query, CachingPolicy cachingPolicy)
   at Sitecore.ExperienceAnalytics.Api.ReportDataService.ExecuteQuery(IReportQueryData queryData, CachingPolicy cachingPolicy)
   at Sitecore.ExperienceAnalytics.Api.ReportingService.RunQuery(ReportQuery reportQuery)
   at Sitecore.ExperienceAnalytics.Api.Http.AnalyticsDataController.Get(ReportQuery reportQuery)


After further investigation found out the Reporting keys were not aligned between CM and Reporting server in the ConnectionStrings.config file.

<add name="reporting.apikey" connectionString="84090dc97055b10532a054da067df368" />

Further reading