Building with CDP/Personalize on Next.js +XM/XP

> Important considerations and learnings
Cover Image for Building with CDP/Personalize on Next.js +XM/XP

Summary

In this post, I will share my general thoughts after having spent some time with CDP and Personalize. I will connect some dots to supplement the official documentation and go a little deeper to discuss some key questions and learnings along the way.

While this post discusses concepts which are applicable to any stack, code examples will focus on a Next.js app using the Engage SDK package with server-set cookies on XM/XP implementations.

I recommend checking out the official docs that cover how the integration works.

Otherwise, let's dive in.

Deep Learning

Concepts

The following concepts and keywords are covered repeatedly throughout this post:

  • Experiments / experiences - I refer to these broadly as "flows"
  • Web vs. full stack flows: Web vs. Interactive/Triggered flows
  • Tracking: sending events (VIEW, IDENTITY, etc.) to or getting data from CDP
  • Personalizing / personalization

Personalize - Web Experiments/Experiences

The quickest way to understand what web flows are and what they are useful for is to look at the OOB templates that ship with Personalize:

Out-of-the-box Personalize Web Flow Templates

Notice how they are all relatively independent of the main parts of the DOM. Popups, sidebars, widgets, alert bars. They are "islands." These types of components are good candidates for web flows because they:

  • Are relatively simple
  • Are unlikely to interfere with code defined in your head application
  • Can be built and shipped quickly
  • Aren't negatively affected by flicker because they are all somewhat expected to appear after the initial page paint

Theoretically, you can build anything you want with web flows because they allow you to inject HTML, CSS, and JavaScript. However, if you want to personalize components that are defined in your head application, you are likely going to want to use full stack (Interactive) flows rather than just web flows.

Pages Router vs App Router

Next.js supports two routing paradigms: the Pages Router (the traditional pages/ directory) and the App Router (the newer app/ directory with React Server Components).

Sitecore's new Content SDK for Next.js supports App Router, but Content SDK is only compatible with SitecoreAI (previously known as XM Cloud).

For now, XM/XP implementations will continue to use the Pages Router (by virtue of the fact that the JSS SDK only supports Pages Router). There is an interesting discussion to keep an eye on. Will Sitecore ever add App Router support to the JSS SDK? How long will Vercel support the Pages Router? Will the Pages Router be deprecated in favor of the App Router?

Server-Side or Bust

The Sitecore docs on how to integrate a Next.js app with Engage SDK (server-set cookies) seem to give client-side integration details more weight than I would have expected. Certainly there is a place for integrating on the client-side, particularly if you want to run web flows, but generally when we are working with head apps such as Next.js, a full stack approach will be more appropriate for enterprise use cases.

The benefits of server-side tracking are clear:

  • Increases data security - you can handle sensitive data on the server without having to expose it on the client side.
  • Increases data flexibility - on the server, you can extend the data, integrate it with other external systems, or otherwise customize it before forwarding it to Sitecore CDP.
  • Prevents ad blockers and web browsers from blocking the tracking code or otherwise manipulating the data.
  • Improves website performance by reducing the number of network requests to external systems. This improves website speed and ensures that data is captured even on poor network connections.

If you want a seamless and bulletproof implementation of CDP/Personalize, client-side will take a back seat to server-side. Client-side anything has numerous problems/flaws, one of them being that when you send behavioral and transactional data from the client-side of your app (via OOB functions), you are sending it directly to Sitecore API endpoints rather than through your server first.

Server-Side or Bust

Server-Side in Practice

In a Next.js app, all requests that match your path matcher config will be processed by your middleware.

This is the place to initialize the Engage SDK, get/set cookies, and potentially fire events (in this case, VIEW).

middleware.ts

_37
import { initServer } from '@sitecore/engage';
_37
import { NextResponse, NextRequest } from 'next/server';
_37
_37
export const config = {
_37
// https://nextjs.org/docs/app/api-reference/file-conventions/middleware#matcher
_37
}
_37
_37
export async function middleware(request: NextRequest) {
_37
_37
const response = NextResponse.next();
_37
_37
const engageSettings = {
_37
clientKey: process.env.NEXT_PUBLIC_CDP_CLIENT_KEY || '',
_37
targetURL: process.env.NEXT_PUBLIC_CDP_TARGET_URL || '',
_37
pointOfSale: process.env.NEXT_PUBLIC_CDP_POINT_OF_SALE || '',
_37
cookieDomain: process.env.NEXT_PUBLIC_CDP_COOKIE_DOMAIN || 'localhost',
_37
cookieExpiryDays: 365,
_37
forceServerCookieMode: true
_37
};
_37
_37
// See https://doc.sitecore.com/cdp/en/developers/api/cookies.html#checking-cookie-consent
_37
const shouldInitEngage = hasUserConsented() && customAuthoringModeCheck() && otherChecks();
_37
_37
// TODO: look at Sitecore's example code for how to handle various states such as edit/preview mode
_37
_37
if (shouldInitEngage){
_37
// Initializes Sitecore Engage server-side library with methods for handling event tracking, identity management, page views, personalization, and browser ID cookie management (getting browser ID from requests, setting browser ID in responses)
_37
const engageServer = initServer(engageSettings);
_37
_37
// This makes an actual HTTP call to Sitecore CDP/Personalize APIs
_37
await engageServer.handleCookie(request, response);
_37
}
_37
_37
// TODO: fire VIEW events for the first page load -- route change events can call another endpoint or you can somehow detect a request to static page assets here as well
_37
_37
return response;
_37
}

Geolocation

Engage/Cloud SDK automatically collect session-based geo data such as city, region, country, and continent based on the user's IP address. In addition to being automatically populated in CDP data, some of the geo data is also accessible in Personalize via OOB conditions.

Geo information is available because user requests to Sitecore's API ostensibly route through Cloudflare. Cloudflare modifies requests to include the geo in request headers such as cf-ipcountry. We can inspect and work with those requests/headers in Personalize.

Sitecore Personalize has some OOB ways of leveraging geo data such as continent, country, and region (only US states at this time). Otherwise, if you need to work with geo that is at the city-level or in "regions" that are outside the USA (provinces, etc.), additional setup is required in Personalize (custom code/conditions). However, you need to create custom JS modules to get it from session.

Example of a custom JS module that can be used to get geo data from session:

Personalize Custom JS Module

Personalize Custom JS Module 2

Example of a custom condition that can be used to check the user's city (keep in mind that you also need to factor in the state):

Personalize Custom City Condition

Personalize Custom City Condition 2

You can create segments in CDP based on session geo data:

CDP Segment Session Geo

Notice how region is not implemented OOB -- you will need to write custom SQL for that. Luckily it's easy to start with an OOB query and update the field it pulls from:


_7
select g.meta_ref as guest_ref
_7
from (select meta_guest_ref
_7
from your_instance.sessions
_7
where upper(your_instance.sessions.type) = upper('WEB') and upper(cast(your_instance.sessions.bxt['geoLocationRegion'] as varchar)) = upper('Texas')) as s
_7
inner join your_instance.guests as g
_7
on s.meta_guest_ref = g.meta_ref
_7
group by g.meta_ref

The one OOB geolocation condition is for US states only:

Personalize OOB Geo Condition

Flicker

Any time you are running personalization on the client side, you are going to experience flicker. The page was painted, and then a call to personalize injects/updates the page, hence the flicker.

In this example component in the XMC official JS starter repo, Sitecore gets and sets an isLoading flag to toggle a placeholder element while the component personalization fetch is in flight:

HeroImageBackground.dev.tsx

_31
export const HeroImageBackground: React.FC<HeroProps> = (props) => {
_31
const [isLoading, setIsLoading] = useState(false);
_31
_31
// ...
_31
_31
useEffect(() => {
_31
const fetchPersonalizedContent = async () => {
_31
try {
_31
setIsLoading(true);
_31
// ...
_31
} catch (error) {
_31
// ...
_31
} finally {
_31
setIsLoading(false);
_31
}
_31
}, [initialFields.dictionary, isPageEditing]);
_31
_31
// ...
_31
_31
return (
_31
<div>
_31
{isLoading && (
_31
<div className="absolute top-0 right-0 p-2 text-xs">
_31
Loading personalized content...
_31
</div>
_31
)}
_31
_31
// ... other components
_31
</div>
_31
);
_31
};

This is a rudimentary approach, but it's a step in the right direction.

  • Suspense states
  • Hide personalized elements areas until
  • Decrease latency as much as possible with pre-fetch, loading scripts as early as possible, etc
  • Use server-side rendering

Additional Considerations

  • Think about scenarios in which CDP/P should be disabled. I did most of the hard work for you here.
  • Check out the official XMC js starter kit for code snippets mentioning cdp and personalize.

Keep it personal,

-MG


More Stories

Cover Image for Script: Boost SIF Certificate Expiry Days

Script: Boost SIF Certificate Expiry Days

> One simple script that definitely won't delete your system32 folder

Cover Image for On Sitecore Stack Exchange (SSE)

On Sitecore Stack Exchange (SSE)

> What I've learned, what I see, what I want to see