SPE Script Performance & Troubleshooting

> Script never ends or runs too slow? Get in here.

April 9, 2023

TLDR

Your problem is that your SPE (Sitecore Powershell Extensions) scripts are running slowly or hanging indefinitely. You are running complex scripts on a large site with a large database. You are seeing many cache clear warnings in the log.

If you have RAM to spare on your local machine, this guide is for you. If not, I recommend searching for items via index rather than via database (if possible, depending on what your script does).

If you're short on time and feeling CRAZY, disable cache limits completely. This is not recommended except on your local machine, but it will definitely improve performance. It's also a great way to see just how big your caches get while running your script.

<settings>
  <setting name="Caching.DisableCacheSizeLimits" value="true"/>
</settings>

Intro

Sitecore cache tuning has been covered in depth by many others, but I want to share my experience with SPE scripts and how I was able to improve performance and resolve issues.

Let's say you have a huge site with a huge database and want to run this script in SPE:

$startPath = "/sitecore/content"
Write-Host "Scanning $($startPath)..."
$targetTemplateId = "{xxxxxxx-xxxxx-xxxxx-xxxxxx-xxxxxxxxx}"
$targetFieldName = "My Field"
$items = Get-ChildItem -Path $startPath -Recurse | Where-Object {
  $_.TemplateId -eq $targetTemplateId -and
	![string]::IsNullOrEmpty($_.Fields[$targetFieldName])
}
Write-Host "Found $($items.Count) matching items"
if(!$items.Count){
  Write-Host "No matching items found. Exiting."
  exit
}
Write-Host "Beginning to loop..."
foreach ($item in $items) {
  Write-Host "Item $($item.ID) $($item.Name) field $($targetFieldName) value: $($item.Fields[$targetFieldName])"
}

The script hangs forever. Logs are filled with variations of:

WARN cache is cleared by Sitecore.Caching.Generics.Cache 1+DefaultScavengeStrategy[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] strategy. Cache running size was 45 MB.

Note that log messages about AccessResultCache are not applicable here.

Head over to /sitecore/admin/cache.aspx. The default UI is a bit clunky, but there may be a module out there that makes it easier to use. Keep refreshing to see which caches are full and adjust them upwards in the config below. If running script on master DB, search for master on cache page. You will notice that as you run your script, the caches will fill up quickly. Keep a close eye on it to see which caches get cleared while your script is running (also check logs for keyword cleared).

<?xml version="1.0" encoding="utf-8" ?>
<sitecore database="SqlServer" xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:security="http://www.sitecore.net/xmlconfig/security/">
  <databases>
    <database id="master" singleInstance="true" type="Sitecore.Data.DefaultDatabase, Sitecore.Kernel" role:require="Standalone or Reporting or Processing or ContentManagement">
      <cacheSizes hint="setting">
        <!-- Increase these -->
        <data>400MB</data>
        <items>500MB</items>
        <paths>5000KB</paths>
        <itempaths>200MB</itempaths>
        <standardValues>5000KB</standardValues>
      </cacheSizes>
    </database>
  </databases>
</sitecore>

Master.config

<?xml version="1.0" encoding="utf-8" ?>
<!-- App_Config\Prefetch\Master.config -->
<configuration>
	<!-- Original value -->
  <!--<cacheSize>200MB</cacheSize>-->
  <!-- New value that worked for me -->
  <cacheSize>600MB</cacheSize>
</configuration>

Increasing your cache size will noticeably increase your script performance and prevent scripts from hanging indefinitely. It will also improve Sitecore performance in general, although it can increase startup time.

Alternatively, if you're feeling CRAZY, you can disable the cache completely. This is not recommended except on your local machine, but it will definitely improve performance. It's also a great way to see just how big your caches get while running your script.

<settings>
  <setting name="Caching.DisableCacheSizeLimits" value="true"/>
</settings>

SPE Script Performance & Troubleshooting

TLDR

Intro

More Reading

More Stories