Installing Sitecore Packages to Containers

When using containers to run Sitecore XP, the way you do some things on the platform are just a little bit different. One of the biggest changes is with how you install packages into Sitecore.

Typically you would use the control panel to upload and install a Sitecore zip package where the application would install any files to the web root folder and any items to the Sitecore database. With containers, this becomes a touch more difficult because:

  1. Container images are intended to immutable and instances are intended to be disposable. That is, the idea is that you can spin down a container, kill it, and spin up a brand new instance from the image and you should be back in business. If you’re manually installing a package after your application starts up from scratch, you’re not taking advantage of the power of containers to its full extent.

  2. Because of #1, Sitecore has locked down runtime access of the application. Specifically, the bin folder (and possibly others) have been made read-only to the runtime user so that you can’t inadventently muck up your application. Therefore, if you try to install a package through the Control Panel interface that adds a .dll file to your bin folder, it’ll fail because the runtime user won’t have access to write to the bin folder.

So how do we install Sitecore packages?

In short, you’ll need to create your own version of the Sitecore images. Think of it like the decorator pattern in software architecture. We’ll basically inherit from the base Sitecore images and make our own. But first, what files are we going to use? And where do the files go?

Container Types

The Sitecore images come in two flavors - we’ll call them runtime images and init images.

Runtime Images

Runtime images are the containers that continue to run during the lifetime of the application - these are your content management, content delivery, processing, publishing, etc - applications that are long-running. Basically, these are the containers that are receiving requests.

Init Images

Init images are special images that run once to do some initialization on the containers once they load. These typically are to handle exceptions for things that can’t be baked into images (or don’t make sense to be baked in). The two that ship with Sitecore are to create the Solr indexes and SQL databases - makes sense in case you’re using a hosted service for either so you’ve got options.

So, if you’re adding/updating files to your runtime images - easy. You’ve got your various files and DLLs to copy in. Hold on to those.

If you’ve got database or search index updates to make, things get a little tricker. For search indexes, we’ll cover that in another blog post. For SQL database updates, we’ll need a DACPAC to update the databases.

A Little Help from an Old Friend

We’ve got our package, but how do we get our asset files and DACPAC from it? The files part is more obvious - we can extract the package itself as it’s nothing more than a regular garden-variety zip file. But the items in the package are serialized .item files. How do we turn those into DACPAC files?

You know what else uses DACPAC files? That’s right, our old friend web deploy packages. Remember them? They’re what used to be cool before containers became a thing for Sitecore. To support web deploy packages, Sitecore has something called the Sitecore Azure Toolkit to help do things like convert Sitecore packages to web deploy packages among other things. Let’s get to work.

Doing the Thing

First, we’ll need to download the Sitecore Azure Toolkit. You can get it here:

https://dev.sitecore.net/Downloads/Sitecore_Azure_Toolkit.aspx

Grab the latest and extract it. You’ll only really need the tools folder but you can extract the whole thing.

We’re going to use the functionality built into the Sitecore.Cloud.PackageConverter.dll and Sitecore.Cloud.Packaging.dll libraries. The following PowerShell will take in the Sitecore package zip file, do the work, and export it to the provided path (with the help of some reflection magic):

# path to the Sitecore package
$path = "C:\MySitecorePackage.zip"
# path to where the converted assets will be output to
$outpath = "C:\MySitecorePackageAssets"
# import the DLL in order to use the types
Add-Type -Path "$PSScriptRoot\Sitecore.Cloud.PackageConverter.dll"
# create a new instance of the PackageConverter object
$packageConverter = New-Object -TypeName "Sitecore.Cloud.PackageConverter.PackageConverter"
# call the ConvertPackage method, this will pull extract the files and convert items to .sql files
$packageConverter.ConvertPackage($path, $outpath)
# create a new instance of the WebDeployPackageBuilder object
$packageBuilder = New-Object -TypeName "Sitecore.Cloud.Packaging.WebDeployPackages.WebDeployPackageBuilder" -ArgumentList @($null)
# get fancy and use reflection to get the ConvertToDacPac method
$convertToDacPac = $packagebuilder.GetType().GetMethod("ConvertToDacPac", [System.Reflection.BindingFlags]([System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Instance))
# go get all the output .sql files and...
Get-ChildItem $outpath -Filter "*.sql" | % {
  # convert them to a dacpac and save out as a file
	$stream = $convertToDacPac.Invoke($packagebuilder, $_.FullName)
	$stream.Seek(0, [System.IO.SeekOrigin]::Begin) | Out-Null
	$filestream = [System.IO.File]::Create("$outpath\$($_.Name.Replace(".sql", ".dacpac"))")
	$stream.CopyTo($filestream)
	$filestream.Flush()
	$filestream.Close()
}

Add-Type -Path "$PSScriptRoot\Sitecore.Cloud.PackageConverter.dll"
# create a new instance of the PackageConverter object
$packageConverter = New-Object -TypeName "Sitecore.Cloud.PackageConverter.PackageConverter"
# call the ConvertPackage method
$packageConverter.ConvertPackage($path, $outpath)

Now you should get several files in your output folder:

  • website: these files will need to go into your web root
  • core.sql/master.sql: these are the SQL statements to update the corresponding databases
  • core.dacpac/master.dacpac: these are the DACPAC files created from their corresponding SQL files

Okay, we’ve got all that we need to update the images. Let’s create some Dockerfiles.

Optional but a good idea: create an asset image

Since you’ve done all this work to get your files, you should create an asset image to be reused. This allows you to recreate your cm and mssql-init images if things change without having to go through this whole business again. The asset image is nothing more than an image that holds all the files you just extracted. Typically (and the way the Sitecore asset images are built) the structure looks like this:

└── module
    ├── cm
    │   └── content
    │       ├── App_Config
    │       │   └── include
    │       │       └── Something.config
    │       ├── bin
    │       │   └── Something.dll
    │       └── Web.config.xdt
    ├── db
    │   ├── Sitecore.Core.dacpac
    │   └── Sitecore.Master.dacpac
    └── tools
        └── Initialize-Config.ps1

There’s a root module folder with separate cm, db, and tools folders under it. The cm folder has a content folder that contains all of the files that need to be copied to the web root. The db folder contains the DACPAC files that will be copied into the mssql-init folder. You typically want to name these to match the format of your database names - so in this case, it’ll be Sitecore.Core.dacpac and Sitecore.Master.dacpac for the Sitecore.Core and Sitecore.Master databases.

Creating New Images

We’ll need to create new images for cm and mssql-init with our updated files:

Content Management

Our content management image will need the files in our website folder. The Dockerfile is pretty straightforward:

ARG ASSETS_IMAGE
ARG CM_IMAGE
FROM ${ASSETS_IMAGE} as assets
FROM ${CM_IMAGE}
COPY --from=assets /module/cm/content /inetpub/wwwroot
...

Copy the files over into your web root and that’s all that’s needed!

SQL Init

The SQL init image is almost as straightforward.

ARG ASSETS_IMAGE
ARG MSSQL_INIT_IMAGE
FROM ${ASSETS_IMAGE} as assets
FROM ${MSSQL_INIT_IMAGE}
COPY --from=assets /module/db /modules/assets
...

This will put your DACPACs into a module folder to deploy into. When you run your container, if you specify the RESOURCES_PATH environment variable to point to C:\modules and set the DATABASES_TO_DEPLOY environment variable to be assets (or whatever you named your directory inside the modules directory), it will pick up all of the dacpac files and deploy them to the appropriate database.

Now, you can run your regular Sitecore mssql-init container and then run your custom asset mssql-init container to make the updates for the package to your database server.

Wrapping Up

That’s it! Now you’ve taken your package, extracted the files needed, converted any item updates to SQL (and then DACPAC) files, created an asset image, and applied the asset images to your base images. Phew, you survived. The good thing is that you if you do this once, you won’t have to do it again and you can reuse all this for other Sitecore packages. Additionally, Sitecore ships most (ahem blob storage) of the first-party packages as asset images already so you don’t have to do all this.

If you’re creating packages of your own items to deploy, I would strongly recommend you look into the items as resources plugin for the Sitecore CLI - instead of going through all the trouble of making asset images, you can create protobuf files that contain all your items and bake them directly into your container image.