Sitecore Docker – Run CM sites on HTTPS


While working on the integrating Sitecore’s CM’s functionality with third party Digital Asset Management(DAM) system, I came across one interesting issue where I wanted my container applications to run on https.

The Issue

After integrating the third party DAM with Sitecore CMS, it found out that, while using local website on developer machine, the developer is not able to load the SSO login page. While loading that page below error messages logged in browser’s console.

Access to the WebCrypto API is restricted to secure origins. Compact View requires HTTPS when used outside localhost (for development).

As clearly stated I am not using localhost as host to access my cm and seems like CM needs to be securely severed over https where as we, for local development, use non secure protocol http.

The Fix

The solution is simple looking form 35,000 feet. Needs CM urls to be serving content over the secure HTTPS protocol. But when actually started digging in details, it is fun and very learning experience.

Performed following steps to achieve this on our local docker development environment.

  1. Clone or download below repository

https://github.com/michaellwest/docker-https

  1. Open the startup/createcert.ps1 for editing and change following parameter’s default value
  • $certificatepassword – from b to more secured passowrd string
  • $dnsNameList – by default the value is *.dev.local. Change this value to match your host pattern. You can specify individual host comma(,) separated.

Note

Alternatively, you can can pass those two as parameter while firing createcert.ps1 command.

3. Open docker-compose.yml file and navigate to cm service and perform following changes:

  • Add new environment parameter HOST_HEADER and set cm host value to it. If you have multiple host(this is what in my case), specify them semicolon(;) separated list. For example host1.dev.local;host2.dev.local
  • Under the volumes bind the new volume for folder startup to c:\startup path of container.
  • Under the port bind the https’s default secure port 443 with the next available container port

Note

The container host port needs to be new port which is not been use by any other services.
  • Lastly, change the entypoint to the path of startup.ps1 file under the startup folder bound under the volumes in above step for startup folder.

After all above changes to CM service of docker-compose.yml file will look something like below

services:
  ...
  cm:
    image: ${REGISTRY}sitecore-xm-cm:${SITECORE_VERSION}-windowsservercore-${WINDOWSSERVERCORE_VERSION}
    entrypoint: powershell.exe -NoLogo -NoProfile -File C:\\startup\\startup.ps1
    volumes:
      - .\src:C:\src
      - .\startup:C:\startup
    ports:
      - "44001:80"
      - "44002:443"
    networks:
      default:
        aliases:
          - cm.dev.local
          ...
    environment:
      HOST_HEADER: cm.dev.local
      ...

4. Copy folder startup and it’s content to same folder where your docker-compose.yml file is.

5. Open the powershell in elevated mode and execute following command

PS> cd <<docker-compose.yml file's folder path>>
PS> ./startup/createcert.ps1

Above powershell script will generate three files cert.cer, cert.pfx and cert.password.text under the startup folder.

This command will also generate the self-signed wildcard certificate and install to your system’s local certificate store under personal.

To verify that run mmc and open local computer certificate store.

Self signed wildcard certificate for *.dev.local

6. Now, is the time to fire up the container for sitecore. To do so run following docker compose command

PS> docker-compose up -d

After successful execution the container services will be up and one can access the host binding with https. For example https://cm.dev.local in this case.

Advance trouble shooting

  1. To check the certificate is exist after the createcert.ps1 script execution. This command can be run on both your local and against cm container as well.
PS> Get-ChildItem -Path cert:\LocalMachine\My
  1. To verify the binding under the container’s IIS for the specified hosts use following commands
PS> Get-Website -Name 'Default Web Site'
  1. To get all the bindins and their port details for a website
PS> Get-WebBinding -Name 'Default Web Site'

References

Sitecore Powershell Reports – Export Functionality not working


The QA team, we are working with has raised a very interesting issue while testing newly created data report using Sitecore Poershell Extenstion(SPE).

The Issue

The data report was executing perfectly but while trying to export the data in any of pre-defined formats offered by SPE, it was not working.

The Sitecore Powershell is allowing export of report data in various industrial standard formats out of the box as listed below:

  • CSV
  • HTML
  • XML
  • Excel
  • JSON

These formats have option links in the ribbon above but when clicking, none of those producing any output.

¯\_(ツ)_/¯

The Investigation

First step is to try to reproduce the issue. We tried on all the available environments and it is happening on every environment except for the local environment.

I though the issue may have to do with some missing config or SPE. So, we quickly try to re-install the SPE module on CM with no over-write option for both files and content on Integration Environment. But that does not fix the issue. ☹

So, we started listing what are the major difference from modules and features point of view between local development and other environments on Azure. It turns out that all other environments have exactly the same piece of code, configurations and modules installed except for Federated Authentication using Sitecore Identity Server for CM.

The Fix

We did quick search over the internet and one of my colleague found out that there is one file which is disabled for SPE which needs to be enable if we are using sitecore 9.1 and higher(we are running 9.3) along with identity server.

The file needs to be enabled is Spe.IdentityServer.config file which is part of the SPE package.

Once package is installed, the file is located at path <<website Root>>\App_Config\Include\Spe\Spe.IdentityServer.config.disabled. The content of this disabled file is as below.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:security="http://www.sitecore.net/xmlconfig/security/">
  <sitecore role:require="Standalone or ContentManagement" security:require="Sitecore">
    <pipelines>
      <owin.cookieAuthentication.validateIdentity>
        <processor type="Sitecore.Owin.Authentication.Pipelines.CookieAuthentication.ValidateIdentity.ValidateSiteNeutralPaths, Sitecore.Owin.Authentication">
          <siteNeutralPaths hint="list">
            <!-- This entry corrects the infinite loop of ExecuteCommand in the SPE Console -->
            <path hint="spe">/sitecore%20modules/PowerShell</path>
          </siteNeutralPaths>
        </processor>
      </owin.cookieAuthentication.validateIdentity>
    </pipelines>
  </sitecore>
</configuration>

Tip: The best way to get these config enable in Docker container world, is to leave the above mentioned .disabled file as is. Create new configuration file with above content in Visual Studio solution deploying to z folder under app_config\include on your binding volume path which will be taken care by Sitecore config patching.

Enjoy exporting SPE reports data…!!!

References

https://github.com/SitecorePowerShell/Console/releases
https://doc.sitecorepowershell.com/troubleshooting

Sitecore Docker -Coveo Diagnosis Page Search Rest Endpoint Issue


While configuring Coveo search on my local development environment using Docker, something got my attention on Coveo’s diagnosis page(<<Sitecore CM Host>>/sitecore modules/Web/Coveo/Admin/CoveoDiagnosticPage.aspx).

This page has all the green ticks except for the one. This is Coveo Search REST Endpoint. See image below for more details.

Coveo Diagnostic Page REST API Error
Coveo Diagnostic Page REST API Error on Docker

As usually, I copied the API end point URL and hit in the Browser and it serve the request with no error. It is HTTP-200 response with valid JSON result.

The error in the log file also not giving any lead either.

Coveo.SearchProvider.Applications.BaseVerifier.VerifyComponent(:0) ERROR StateVerifier has encountered an error while verifying the component "Coveo Search REST Endpoint".<br>Exception: Coveo.SearchProvider.Applications.StateVerifierException<br>Message: Failed to obtain resource located at 'http://sicore930.dev.local/coveo/rest/'. See inner exception for details.<br>Source: Coveo.SearchProvider.Applications<br>at Coveo.SearchProvider.Applications.CommonStateVerifier.TryPingRestEndpoint(String p_Endpoint)<br>at Coveo.SearchProvider.Applications.CommonStateVerifier.<>c__DisplayClass11_0.b__0()<br>at Coveo.SearchProvider.Applications.BaseVerifier.VerifyComponent(Func`1 p_VerifyMethod, String p_ComponentName)

I opened docker powershell command for the cm container and tried to request the host from within container.

PS> Docker container ls

Grab the name or container id

PS> Docker exec -it <<container name/id>> cmd

This will open a powershell window connected to the container specified.

PS> Invoke-RestMethod -Uri "http://sitecore930.dev.local/coveo/rest" | ConvertTo-Json

It has given me following error

Now things are little clear to me. The Coveo diagnosis page is using current host to make REST API request from IIS server running on the container. The container does not have host file entry to reseolve the host url.

So now, I started searching how to make entry in container’s host file??? I was sure, I am not the first person who wanted this kind of stuff.

Following PoweShell command has does the trick.

PS> echo 127.0.0.1 sitecore930.dev.local >> "C:\Windows\System32\drivers\etc\hosts"

After executing above command on Container’s powershell, now my Coveo Diagnosis page is Happy…!!!

Please do let me know, is there any better solution you guys have in place for this issue.

Till that time happy Coveo searching…!!!

Sitecore Docker – SQL Databse connection issue


Psssss… This is more as a note to myself..!!!

While setting my local environment for development on Sitecore 9.3 using docker, I came across below issue.

Docker - The network path was not found
Docker SQL Connection Issue
[Win32Exception (0x80004005): The network path was not found]

[SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)]
   System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection) +1341
   System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) +159
   System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) +382
   System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) +307
   System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry) +198
   System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry) +422
   System.Data.SqlClient.SqlConnection.Open() +199
   Sitecore.Data.DataProviders.Sql.DataProviderCommand..ctor(IDbCommand command, DataProviderTransaction transaction, Boolean openConnection) +113
   Sitecore.Data.DataProviders.Sql.<>c__DisplayClass26_0.<CreateCommand>b__0() +48
   Sitecore.Data.DataProviders.NullRetryer.Execute(Func`1 action, Action recover) +293
   Sitecore.Data.DataProviders.Sql.<>c__DisplayClass29_0.<CreateReader>b__0() +30
   Sitecore.Data.DataProviders.NullRetryer.Execute(Func`1 action, Action recover) +293
   Sitecore.Data.DataProviders.Sql.SqlDataApi.CreateReader(String sql, Object[] parameters) +281
   Sitecore.Data.DataProviders.Sql.SqlDataProvider.GetContentLanguages() +169
   Sitecore.Data.DataProviders.Sql.SqlDataProvider.LoadLanguages() +133
   Sitecore.Data.DataProviders.Sql.SqlDataProvider.GetLanguages() +49
   Sitecore.Data.SqlServer.SqlServerDataProvider.ExecutePreLoadItemDefinitionSql(String sql, Object[] parameters, SafeDictionary`2 prefetchData) +52
   Sitecore.Data.DataProviders.Retryer.ExecuteNoResult(Action action, Action recover) +539
   Sitecore.Data.SqlServer.SqlServerDataProvider.LoadInitialItemDefinitions(String condition, Object[] parameters, SafeDictionary`2 prefetchData) +237
   Sitecore.Data.DataProviders.Sql.SqlDataProvider.EnsureInitialPrefetch() +333
   Sitecore.Data.DataProviders.Sql.SqlDataProvider.GetPrefetchData(ID itemId) +62
   Unicorn.Data.DataProvider.UnicornSqlServerDataProvider.GetItemDefinition(ID itemId, CallContext context) +242
   Sitecore.Data.DataProviders.DataProvider.GetItemDefinition(ID itemID, CallContext context, DataProviderCollection providers) +156
   Sitecore.Data.DataSource.GetItemInformation(ID itemID) +88
   Sitecore.Data.DataSource.GetItemData(ID itemID, Language language, Version version) +32
   Sitecore.Data.Engines.TemplateEngine.GetdefaultSectionOrder() +118
   Sitecore.Data.Engines.TemplateEngine.InternalGetTemplates() +486
   Sitecore.Data.Engines.TemplateEngine.GetTemplate(ID templateId) +184
   Sitecore.XA.Foundation.SitecoreExtensions.Extensions.DatabaseExtensions.GetContentItemsOfTemplate(Database database, ID templateId) +126
   Sitecore.XA.Foundation.Multisite.SiteResolvers.EnvironmentSitesResolver.ResolveAllSites(Database database) +63
   Sitecore.XA.Foundation.Multisite.Providers.SxaSiteProvider.GetSiteList() +162
   Sitecore.XA.Foundation.Multisite.Providers.SxaSiteProvider.InitializeSites() +105
   Sitecore.XA.Foundation.Multisite.Providers.SxaSiteProvider.GetSites() +18
   System.Linq.<SelectManyIterator>d__17`2.MoveNext() +265
   Sitecore.Sites.SiteCollection.AddRange(IEnumerable`1 sites) +221
   Sitecore.Sites.SitecoreSiteProvider.GetSites() +258
   Sitecore.Sites.DefaultSiteContextFactory.GetSites() +253
   Sitecore.XA.Foundation.Multisite.SiteInfoResolver.get_Sites() +60
   Sitecore.XA.Foundation.Multisite.Pipelines.Initialize.InitSiteManager.Process(PipelineArgs args) +85
   (Object , Object ) +9
   Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) +490
   Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain, Boolean failIfNotExists) +236
   Sitecore.Pipelines.DefaultCorePipelineManager.Run(String pipelineName, PipelineArgs args, String pipelineDomain) +22
   Sitecore.Nexus.Web.HttpModule.Application_Start() +220
   Sitecore.Nexus.Web.HttpModule.Init(HttpApplication app) +1165
   System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers) +584
   System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context) +168
   System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context) +277
   System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext) +369

[HttpException (0x80004005): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)]
   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +532
   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +111
   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +724

The error speaks itself, that the CM and/or CD was not able to connect to underlying SQL server. SO, I immediately opens the SSMS and try to connect to my SQL server running as container on my local and I am successfully able to connect both using IP address as well as using alias(in my case it’s sql).

So what’s the issue??? When I am able to connect to SQL server running in container using outside of the Docker created network than why my CM and/or CD instance is not able to connect to it within the same network???

I check inward and outward binding ports setting in docker-compose.yml file for SQL service. Double check connection strings for server name, user name and password and that all looks correct.

Thanks to one of my colleague, who points out my docker-compose file was missing hostname for SQL server service.

I have added them to my docker-compose.yml for SQL, SOLR and XCONNECT and happy days.

The sample docker-compose.yml is as below.

version: '2.4'

networks:
  dev.local:

services:

  sql:
    hostname: sql
    image: ${REGISTRY}sitecore-xp-sxa-coveo-def-sqldev:${SITECORE_VERSION}-windowsservercore-${WINDOWSSERVERCORE_VERSION}
    networks:
      dev.local:   
    volumes:
      - .\data\sql:C:\Data
    mem_limit: 2GB
   
    ports:
      - "44010:1433"
    environment:
      SA_PASSWORD: ${SQL_SA_PASSWORD}
      ACCEPT_EULA: "Y"

  solr:
    hostname: solr
    image: ${REGISTRY}sitecore-xp-sxa-solr:${SITECORE_VERSION}-nanoserver-${NANOSERVER_VERSION}
    networks:
      dev.local:
    volumes:
      - .\data\solr:C:\Data
    mem_limit: 1GB
    networks:
      dev.local:
    ports:
      - "44011:8983"

  xconnect:
    hostname: xconnect
    image: ${REGISTRY}sitecore-xp-xconnect:${SITECORE_VERSION}-windowsservercore-${WINDOWSSERVERCORE_VERSION}
    volumes:
      - .\data\xconnect\logs:C:\inetpub\wwwroot\App_Data\logs
      - .\data\xconnect-appdata\models:C:\inetpub\wwwroot\App_Data\Models
    networks:
      dev.local:
    ports:
        - "44012:80"
    mem_limit: 1GB
    environment:
      SITECORE_LICENSE: ${SITECORE_LICENSE}
      SITECORE_SITECORE:XCONNECT:COLLECTIONSEARCH:SERVICES:SOLR.SOLRREADERSETTINGS:OPTIONS:REQUIREHTTPS: 'false'
      SITECORE_SITECORE:XCONNECT:SEARCHINDEXER:SERVICES:SOLR.SOLRWRITERSETTINGS:OPTIONS:REQUIREHTTPS: 'false'
      SITECORE_CONNECTIONSTRINGS_MESSAGING: Data Source=sql;Database=Sitecore.Messaging;User ID=sa;Password=${SQL_SA_PASSWORD}
      SITECORE_CONNECTIONSTRINGS_PROCESSING.ENGINE.STORAGE: Data Source=sql;Database=Sitecore.ProcessingEngineStorage;User ID=sa;Password=${SQL_SA_PASSWORD}
      SITECORE_CONNECTIONSTRINGS_REPORTING: Data Source=sql;Database=Sitecore.Reporting;User ID=sa;Password=${SQL_SA_PASSWORD}
      SITECORE_CONNECTIONSTRINGS_XDB.MARKETINGAUTOMATION: Data Source=sql;Database=Sitecore.MarketingAutomation;User ID=sa;Password=${SQL_SA_PASSWORD}
      SITECORE_CONNECTIONSTRINGS_XDB.PROCESSING.POOLS: Data Source=sql;Database=Sitecore.Processing.Pools;User ID=sa;Password=${SQL_SA_PASSWORD}
      SITECORE_CONNECTIONSTRINGS_XDB.REFERENCEDATA: Data Source=sql;Database=Sitecore.ReferenceData;User ID=sa;Password=${SQL_SA_PASSWORD}
      SITECORE_CONNECTIONSTRINGS_COLLECTION: Data Source=sql;Database=Sitecore.Xdb.Collection.ShardMapManager;User ID=sa;Password=${SQL_SA_PASSWORD}
      SITECORE_CONNECTIONSTRINGS_SOLRCORE: http://solr:8983/solr/sitecore_xdb
    depends_on:
      - sql
      - solr