SPE Script Performance & Troubleshooting

> Script never ends or runs too slow? Get in here.
Cover Image for SPE Script Performance & Troubleshooting

TLDR

Your problem is that your SPE (Sitecore Powershell Extensions) scripts are running slowly or hanging indefinitely. You are running complex scripts on a large site with a large database. You are seeing many cache clear warnings in the log.

If you have RAM to spare on your local machine, this guide is for you. If not, I recommend searching for items via index rather than via database (if possible, depending on what your script does).

If you're short on time and feeling CRAZY, disable cache limits completely. This is not recommended except on your local machine, but it will definitely improve performance. It's also a great way to see just how big your caches get while running your script.


_3
<settings>
_3
<setting name="Caching.DisableCacheSizeLimits" value="true"/>
_3
</settings>

Intro

Sitecore cache tuning has been covered in depth by many others, but I want to share my experience with SPE scripts and how I was able to improve performance and resolve issues.

Let's say you have a huge site with a huge database and want to run this script in SPE:


_24
$startPath = "/sitecore/content"
_24
_24
Write-Host "Scanning $($startPath)..."
_24
_24
$targetTemplateId = "{xxxxxxx-xxxxx-xxxxx-xxxxxx-xxxxxxxxx}"
_24
$targetFieldName = "My Field"
_24
_24
$items = Get-ChildItem -Path $startPath -Recurse | Where-Object {
_24
$_.TemplateId -eq $targetTemplateId -and
_24
![string]::IsNullOrEmpty($_.Fields[$targetFieldName])
_24
}
_24
_24
Write-Host "Found $($items.Count) matching items"
_24
_24
if(!$items.Count){
_24
Write-Host "No matching items found. Exiting."
_24
exit
_24
}
_24
_24
Write-Host "Beginning to loop..."
_24
_24
foreach ($item in $items) {
_24
Write-Host "Item $($item.ID) $($item.Name) field $($targetFieldName) value: $($item.Fields[$targetFieldName])"
_24
}

The script hangs forever. Logs are filled with variations of:

WARN cache is cleared by Sitecore.Caching.Generics.Cache 1+DefaultScavengeStrategy[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]] strategy. Cache running size was 45 MB.

Note that log messages about AccessResultCache are not applicable here.

Head over to /sitecore/admin/cache.aspx. The default UI is a bit clunky, but there may be a module out there that makes it easier to use. Keep refreshing to see which caches are full and adjust them upwards in the config below. If running script on master DB, search for master on cache page. You will notice that as you run your script, the caches will fill up quickly. Keep a close eye on it to see which caches get cleared while your script is running (also check logs for keyword cleared).


_15
<?xml version="1.0" encoding="utf-8" ?>
_15
<sitecore database="SqlServer" xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:security="http://www.sitecore.net/xmlconfig/security/">
_15
<databases>
_15
<database id="master" singleInstance="true" type="Sitecore.Data.DefaultDatabase, Sitecore.Kernel" role:require="Standalone or Reporting or Processing or ContentManagement">
_15
<cacheSizes hint="setting">
_15
<!-- Increase these -->
_15
<data>400MB</data>
_15
<items>500MB</items>
_15
<paths>5000KB</paths>
_15
<itempaths>200MB</itempaths>
_15
<standardValues>5000KB</standardValues>
_15
</cacheSizes>
_15
</database>
_15
</databases>
_15
</sitecore>

Master.config

_9
<?xml version="1.0" encoding="utf-8" ?>
_9
<!-- App_Config\Prefetch\Master.config -->
_9
<configuration>
_9
<!-- Original value -->
_9
<!--<cacheSize>200MB</cacheSize>-->
_9
_9
<!-- New value that worked for me -->
_9
<cacheSize>600MB</cacheSize>
_9
</configuration>

Increasing your cache size will noticeably increase your script performance and prevent scripts from hanging indefinitely. It will also improve Sitecore performance in general, although it can increase startup time.

Alternatively, if you're feeling CRAZY, you can disable the cache completely. This is not recommended except on your local machine, but it will definitely improve performance. It's also a great way to see just how big your caches get while running your script.


_3
<settings>
_3
<setting name="Caching.DisableCacheSizeLimits" value="true"/>
_3
</settings>

More Reading

Cheers,

-MG


More Stories

Cover Image for Ideas For Docker up.ps1 Scripts

Ideas For Docker up.ps1 Scripts

> Because Docker can be brittle

Cover Image for NextJS: Access has been blocked by CORS policy

NextJS: Access has been blocked by CORS policy

> CORS is almost as much of a nuisance as GDPR popups

Cover Image for Super Fast Project Builds with Visual Studio Publish

Super Fast Project Builds with Visual Studio Publish

> For when solution builds take too long

Cover Image for Tips for Forms Implementations

Tips for Forms Implementations

> And other pro tips

Cover Image for Script: Boost SIF Certificate Expiry Days

Script: Boost SIF Certificate Expiry Days

> One simple script that definitely won't delete your system32 folder

Cover Image for Symposium 2022 Reflections

Symposium 2022 Reflections

> Sitecore is making big changes

Cover Image for On Sitecore Stack Exchange (SSE)

On Sitecore Stack Exchange (SSE)

> What I've learned, what I see, what I want to see

Cover Image for NextJS: Unable to Verify the First Certificate

NextJS: Unable to Verify the First Certificate

> UNABLE_TO_VERIFY_LEAF_SIGNATURE

Cover Image for NextJS/JSS Edit Frames Before JSS v21.1.0

NextJS/JSS Edit Frames Before JSS v21.1.0

> It is possible. We have the technology.

Cover Image for JSS: Reducing Bloat in Multilist Field Serialization

JSS: Reducing Bloat in Multilist Field Serialization

> Because: performance, security, and error-avoidance

Cover Image for Tips for Applying Cumulative Sitecore XM/XP Patches and Hotfixes

Tips for Applying Cumulative Sitecore XM/XP Patches and Hotfixes

> It's probably time to overhaul your processes

Cover Image for Content Editor Search Bar Not Working

Content Editor Search Bar Not Working

> Sometimes it works, sometimes not

Cover Image for Security Series: App Service IP Restrictions

Security Series: App Service IP Restrictions

> How to manage IP rules "at scale" using the Azure CLI

Cover Image for On Mentorship and Community Contributions

On Mentorship and Community Contributions

> Reflections and what I learned as an MVP mentor

Cover Image for Azure PaaS Cache Optimization

Azure PaaS Cache Optimization

> App Services benefit greatly from proper configuration

Cover Image for JSS + TypeScript Sitecore Project Tips

JSS + TypeScript Sitecore Project Tips

> New tech, new challenges

Cover Image for Year in Review: 2022

Year in Review: 2022

> Full steam ahead

Cover Image for Troubleshooting 502 Responses in Azure App Services

Troubleshooting 502 Responses in Azure App Services

> App Services don't support all libraries

Cover Image for Critical Security Bulletin SC2024-001-619349 Announced

Critical Security Bulletin SC2024-001-619349 Announced

> And other scintillating commentary

Cover Image for Hello World

Hello World

> Welcome to the show

Cover Image for NextJS: Short URL for Viewing Layout Service Response

NextJS: Short URL for Viewing Layout Service Response

> Because the default URL is 2long4me

Cover Image for On Sitecore Development

On Sitecore Development

> Broadly speaking

Cover Image for Early Returns in React Components

Early Returns in React Components

> When and how should you return early in a React component?

Cover Image for Tips for New Sitecore Developers

Tips for New Sitecore Developers

> If I had more time, I would have written a shorter letter

Cover Image for How to Run Old Versions of Solr in a Docker Container

How to Run Old Versions of Solr in a Docker Container

> Please don't make me install another version of Solr on my local...

Cover Image for Sitecore Symposium 2022

Sitecore Symposium 2022

> What I'm Watching 👀

Cover Image for Don't Ignore the HttpRequestValidationException

Don't Ignore the HttpRequestValidationException

> Doing so could be... potentially dangerous

Cover Image for Add TypeScript Type Checks to RouteData fields

Add TypeScript Type Checks to RouteData fields

> Inspired by error: Conversion of type may be a mistake because neither type sufficiently overlaps with the other.