JSS: Reducing Bloat in Multilist Field Serialization

> Because: performance, security, and error-avoidance
Cover Image for JSS: Reducing Bloat in Multilist Field Serialization

The Problem

I was recently involved in a Slack conversation around the topic of reducing the size of the serialized payload in JSS. The conversation was prompted by a Sitecore Support ticket that was opened by a customer. The customer was experiencing issues with the Experience Editor failing to load when the serialized payload was too large.

The problem can be summarized as follows:

Let's say you have an item with a list field that references multiple items, and the items you are referencing have many fields. The default JSS behavior is to serialize all of those items' fields. You may not want this to be the case, either for performance reasons or because you don't want to expose that information to the client.

Experience Editor Issues

Have you ever seen the JS build logs warn you about serializes pages being too large? Take those warnings seriously; you may want to do some optimization.

If the serialized payload is too large (i.e. larger than 2MB), it can cause the Experience Editor to fail during load. Keep in mind that Experience Editor adds its own bloat to the payload in the form of editable chrome markups.

Sitecore Support recommended the following:

You can mark certain unnecessary fields as non-editable to trim down the serialized payload. To achieve this, you can create custom serializers for such field types extending from the BaseFieldSerializer class. Then you can override the RenderField method and set disableEditing to true for the unnecessary fields based on a condition (for example, field name or field id). The serializers will then need to be added to or replaced in the getFieldSerializer pipeline (under layoutService group) in Sitecore configuration.

One Approach

I propose a different approach that is arguably more powerful. We can override the GetMultilistFieldSerializer with our own. Methods can then be overridden with logic just for the list fields. If the field isn't a list type, we can call the base.

This method allows us to only serialize the fields that we want. By following this guide, you should also be in a good place to "override the RenderField method and set disableEditing to true for the unnecessary fields".

Note that the examples below use dependency injection, which is a pattern I highly recommend for Sitecore solutions. The examples are intentionally incomplete to avoid cluttering the post with the usual minutia of dependency injection configuration.

Specify the Custom GetMultilistFieldSerializer

Project.Main.Config

_11
<pipelines>
_11
<group groupName="layoutService">
_11
<pipelines>
_11
<getFieldSerializer performanceCritical="true">
_11
<processor type="Sitecore.LayoutService.Serialization.Pipelines.GetFieldSerializer.GetMultilistFieldSerializer, Sitecore.LayoutService" resolve="true">
_11
<patch:attribute name="type">Website.Foundation.LayoutService.Serialization.Pipelines.GetFieldSerializer.GetMultilistFieldSerializer, Website</patch:attribute>
_11
</processor>
_11
</getFieldSerializer>
_11
</pipelines>
_11
</group>
_11
</pipelines>

GetMultilistFieldSerializer.cs

_36
using Sitecore.Abstractions;
_36
using Sitecore.Diagnostics;
_36
using Sitecore.LayoutService.Serialization;
_36
using Sitecore.LayoutService.Serialization.Pipelines.GetFieldSerializer;
_36
using Sitecore.Services.GraphQL.EdgeSchema.Services.Multisite;
_36
using Website.Foundation.LayoutService.Serialization.FieldSerializers;
_36
using Website.Foundation.LayoutService.Services;
_36
_36
namespace Website.Foundation.LayoutService.Serialization.Pipelines.GetFieldSerializer
_36
{
_36
public class GetMultilistFieldSerializer : Sitecore.LayoutService.Serialization.Pipelines.GetFieldSerializer.GetMultilistFieldSerializer
_36
{
_36
private readonly BaseMediaManager _mediaManager;
_36
private readonly IMultisiteService _multisiteService;
_36
private readonly ISerializedTargetItemService _serializedTargetItemService;
_36
_36
public GetMultilistFieldSerializer(
_36
IFieldRenderer fieldRenderer,
_36
BaseMediaManager mediaManager,
_36
IMultisiteService multisiteService,
_36
ISerializedTargetItemService serializedTargetItemService) : base(fieldRenderer, mediaManager, multisiteService)
_36
{
_36
_mediaManager = mediaManager;
_36
_multisiteService = multisiteService;
_36
_serializedTargetItemService = serializedTargetItemService;
_36
}
_36
_36
protected override void SetResult(GetFieldSerializerPipelineArgs args)
_36
{
_36
Assert.ArgumentNotNull(args, "args");
_36
Assert.IsNotNull(args.Field, "args.Field is null");
_36
Assert.IsNotNull(args.ItemSerializer, "args.ItemSerializer is null");
_36
args.Result = new MultiListFieldSerializer(args.ItemSerializer, FieldRenderer, _mediaManager, _multisiteService, _serializedTargetItemService);
_36
}
_36
}
_36
}

Add a Custom MultiListFieldSerializer

MultilistFieldSerializer.cs

_37
using System;
_37
using System.Collections.Generic;
_37
using Newtonsoft.Json.Linq;
_37
using Sitecore.Abstractions;
_37
using Sitecore.Data.Fields;
_37
using Sitecore.Data.Items;
_37
using Sitecore.LayoutService.Serialization;
_37
using Sitecore.LayoutService.Serialization.ItemSerializers;
_37
using Sitecore.Links;
_37
using Website.Foundation.LayoutService.Services;
_37
_37
namespace Website.Foundation.LayoutService.Serialization.FieldSerializers
_37
{
_37
public class MultilistFieldSerializer : Sitecore.LayoutService.Serialization.FieldSerializers.MultilistFieldSerializer
_37
{
_37
private readonly ISerializedTargetItemService _serializedTargetItemService;
_37
_37
public MultilistFieldSerializer(
_37
IItemSerializer itemSerializer,
_37
IFieldRenderer fieldRenderer,
_37
BaseMediaManager mediaManager,
_37
ISerializedTargetItemService serializedTargetItemService) : base(itemSerializer, fieldRenderer, mediaManager)
_37
{
_37
_serializedTargetItemService = serializedTargetItemService;
_37
}
_37
_37
protected override string GetSerializedTargetItem(Item item, MultilistField field, int depth)
_37
{
_37
JObject? jObject = _serializedTargetItemService.GetSerializedTargetItemFields(item, depth);
_37
if (jObject != null)
_37
{
_37
return jObject.ToString();
_37
}
_37
return base.GetSerializedTargetItem(item, field, depth);
_37
}
_37
}
_37
}

SerializedItemFieldsRepository.cs

_38
using Newtonsoft.Json.Linq;
_38
using Sitecore.Data.Items;
_38
using Website.Foundation.LayoutService;
_38
using Website.Foundation.LayoutService.Repositories;
_38
_38
namespace Website.Project.Main.Repositories
_38
{
_38
public class SerializedItemFieldsRepository : ISerializedItemFieldsRepository
_38
{
_38
...
_38
_38
public JObject? GetSerializedItemFields(Item item, int depth)
_38
{
_38
// These are random IDs for example purposes
_38
if (item.TemplateID == new ID("{3CCEB664-AAC5-40D5-BB19-7FFF94C619B5}"))
_38
{
_38
return GetFields(item, new[] {
_38
new ID("{0BEB9157-C202-4385-ABB1-0FE2CA2AA0A4}"),
_38
new ID("{AB10475E-D72B-4DD5-B068-24BAAE87A4BD}")
_38
}, depth);
_38
}
_38
_38
return null;
_38
}
_38
_38
private JObject GetFields(Item item, ID[]? fieldIds, int depth)
_38
{
_38
var serializationOptions = new SerializationOptions { DisableEditing = true };
_38
_38
if (fieldIds != null)
_38
{
_38
return JObject.Parse(_itemSerializer.SerializeFields(item, fieldIds, serializationOptions, depth));
_38
}
_38
_38
return JObject.Parse(_itemSerializer.Serialize(item, serializationOptions));
_38
}
_38
}
_38
}

Add a CustomItemSerializer

Project.Main.Config

_11
<layoutService>
_11
<configurations>
_11
<config name="jss">
_11
<rendering type="Sitecore.LayoutService.Configuration.DefaultRenderingConfiguration, Sitecore.LayoutService">
_11
<itemSerializer patch:instead="*[@type='Sitecore.JavaScriptServices.ViewEngine.LayoutService.JssItemSerializer, Sitecore.JavaScriptServices.ViewEngine']" type="Website.Foundation.LayoutService.Serialization.ItemSerializers.CustomItemSerializer, Website" resolve="true">
_11
<AlwaysIncludeEmptyFields>true</AlwaysIncludeEmptyFields>
_11
</itemSerializer>
_11
</rendering>
_11
</config>
_11
</configurations>
_11
</layoutService>

CustomItemSerializer.cs

_41
using System.IO;
_41
using Newtonsoft.Json;
_41
using Sitecore.Data;
_41
using Sitecore.Data.Fields;
_41
using Sitecore.Data.Items;
_41
using Sitecore.JavaScriptServices.ViewEngine.LayoutService;
_41
using Sitecore.LayoutService.Serialization;
_41
using Sitecore.LayoutService.Serialization.Pipelines.GetFieldSerializer;
_41
_41
namespace Website.Foundation.LayoutService.Serialization.ItemSerializers
_41
{
_41
public class CustomItemSerializer : JssItemSerializer, ICustomItemSerializer
_41
{
_41
public CustomItemSerializer(IGetFieldSerializerPipeline getFieldSerializerPipeline) : base(getFieldSerializerPipeline)
_41
{
_41
}
_41
_41
// Add ability to serialize a defined set of fields.
_41
public virtual string SerializeFields(Item item, ID[] fieldIds, SerializationOptions options, int depth)
_41
{
_41
using (StringWriter stringWriter = new StringWriter())
_41
{
_41
using (JsonTextWriter jsonTextWriter = new JsonTextWriter(stringWriter))
_41
{
_41
jsonTextWriter.WriteStartObject();
_41
foreach (ID fieldId in fieldIds)
_41
{
_41
Field? itemField = item.Fields[fieldId];
_41
if (itemField != null)
_41
{
_41
SerializeField(itemField, jsonTextWriter, options, depth);
_41
}
_41
}
_41
_41
jsonTextWriter.WriteEndObject();
_41
}
_41
return stringWriter.ToString();
_41
}
_41
}
_41
}
_41
}

SerializedTargetItemService.cs

_38
using System.Collections.Generic;
_38
using Newtonsoft.Json.Linq;
_38
using Sitecore.Data.Items;
_38
using Website.Foundation.LayoutService.Repositories;
_38
_38
namespace Website.Foundation.LayoutService.Services
_38
{
_38
public class SerializedTargetItemService : ISerializedTargetItemService
_38
{
_38
private readonly IEnumerable<ISerializedItemFieldsRepository> _serializedItemFieldsRepositories;
_38
_38
public SerializedTargetItemService(
_38
IEnumerable<ISerializedItemFieldsRepository> serializedItemFieldsRepositories)
_38
{
_38
_serializedItemFieldsRepositories = serializedItemFieldsRepositories;
_38
}
_38
_38
public JObject? GetSerializedTargetItemFields(Item item, int depth)
_38
{
_38
var jobj = new JObject();
_38
_38
foreach (ISerializedItemFieldsRepository serializedItemFieldsRepository in _serializedItemFieldsRepositories)
_38
{
_38
JObject? contents = serializedItemFieldsRepository.GetSerializedItemFields(item, depth);
_38
if (contents != null)
_38
{
_38
jobj.Merge(contents);
_38
}
_38
}
_38
if (jobj.Count > 0)
_38
{
_38
return jobj;
_38
}
_38
_38
return null;
_38
}
_38
}
_38
}

Another Approach / More Reading

https://stackoverflow.com/questions/70503440/how-to-override-the-4mb-api-routes-body-size-limit/70504864#70504864


_8
5468 11:26:36 ERROR [JSS] Error occurred during POST to remote rendering host: `http://localhost:3000/api/editing/render`
_8
5468 11:26:36 ERROR The remote server returned an error: (413) Body exceeded 2mb limit.
_8
Exception: System.Net.WebException
_8
Message: The remote server returned an error: (413) Body exceeded 2mb limit.
_8
Source: System
_8
at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
_8
at System.Net.WebClient.UploadString(Uri address, String method, String data)
_8
at Sitecore.JavaScriptServices.ViewEngine.Http.RenderEngine.Invoke[T](String moduleName, String functionName, Object[] functionArgs)

See if your solution contains this, and modify the value:


_9
// Bump body size limit (1mb by default) for Sitecore editor payload
_9
// See https://nextjs.org/docs/api-routes/api-middlewares#custom-config
_9
export const config = {
_9
api: {
_9
bodyParser: {
_9
sizeLimit: '2mb',
_9
},
_9
},
_9
};

Keep on building,

Marcel


More Stories

Cover Image for On Mentorship and Community Contributions

On Mentorship and Community Contributions

> Reflections and what I learned as an MVP mentor

Cover Image for Add TypeScript Type Checks to RouteData fields

Add TypeScript Type Checks to RouteData fields

> Inspired by error: Conversion of type may be a mistake because neither type sufficiently overlaps with the other.

Cover Image for Symposium 2022 Reflections

Symposium 2022 Reflections

> Sitecore is making big changes

Cover Image for Azure PaaS Cache Optimization

Azure PaaS Cache Optimization

> App Services benefit greatly from proper configuration

Cover Image for NextJS/JSS Edit Frames Before JSS v21.1.0

NextJS/JSS Edit Frames Before JSS v21.1.0

> It is possible. We have the technology.

Cover Image for Don't Ignore the HttpRequestValidationException

Don't Ignore the HttpRequestValidationException

> Doing so could be... potentially dangerous

Cover Image for NextJS: Unable to Verify the First Certificate

NextJS: Unable to Verify the First Certificate

> UNABLE_TO_VERIFY_LEAF_SIGNATURE

Cover Image for On Sitecore Development

On Sitecore Development

> Broadly speaking

Cover Image for Early Returns in React Components

Early Returns in React Components

> When and how should you return early in a React component?

Cover Image for On Sitecore Stack Exchange (SSE)

On Sitecore Stack Exchange (SSE)

> What I've learned, what I see, what I want to see

Cover Image for Security Series: App Service IP Restrictions

Security Series: App Service IP Restrictions

> How to manage IP rules "at scale" using the Azure CLI

Cover Image for Critical Security Bulletin SC2024-001-619349 Announced

Critical Security Bulletin SC2024-001-619349 Announced

> And other scintillating commentary

Cover Image for NextJS: Short URL for Viewing Layout Service Response

NextJS: Short URL for Viewing Layout Service Response

> Because the default URL is 2long4me

Cover Image for Year in Review: 2022

Year in Review: 2022

> Full steam ahead

Cover Image for Super Fast Project Builds with Visual Studio Publish

Super Fast Project Builds with Visual Studio Publish

> For when solution builds take too long

Cover Image for Ideas For Docker up.ps1 Scripts

Ideas For Docker up.ps1 Scripts

> Because Docker can be brittle

Cover Image for NextJS: Access has been blocked by CORS policy

NextJS: Access has been blocked by CORS policy

> CORS is almost as much of a nuisance as GDPR popups

Cover Image for Tips for Applying Cumulative Sitecore XM/XP Patches and Hotfixes

Tips for Applying Cumulative Sitecore XM/XP Patches and Hotfixes

> It's probably time to overhaul your processes

Cover Image for Hello World

Hello World

> Welcome to the show

Cover Image for Script: Boost SIF Certificate Expiry Days

Script: Boost SIF Certificate Expiry Days

> One simple script that definitely won't delete your system32 folder

Cover Image for SPE Script Performance & Troubleshooting

SPE Script Performance & Troubleshooting

> Script never ends or runs too slow? Get in here.

Cover Image for Tips for Forms Implementations

Tips for Forms Implementations

> And other pro tips

Cover Image for How to Run Old Versions of Solr in a Docker Container

How to Run Old Versions of Solr in a Docker Container

> Please don't make me install another version of Solr on my local...

Cover Image for Troubleshooting 502 Responses in Azure App Services

Troubleshooting 502 Responses in Azure App Services

> App Services don't support all libraries

Cover Image for Content Editor Search Bar Not Working

Content Editor Search Bar Not Working

> Sometimes it works, sometimes not

Cover Image for JSS + TypeScript Sitecore Project Tips

JSS + TypeScript Sitecore Project Tips

> New tech, new challenges

Cover Image for Tips for New Sitecore Developers

Tips for New Sitecore Developers

> If I had more time, I would have written a shorter letter

Cover Image for Sitecore Symposium 2022

Sitecore Symposium 2022

> What I'm Watching 👀