xConnect: Storing Full URLs in Page View Events

October 9, 2022

Index

Overview
1. Override the Default Page View Event Creation Logic
2. Create a Custom Event That Stores the Full URL
More Reading

Overview

Data lakes, warehouses, swamps, and graveyards are all the rage these days. Imagine a scenario in which you want to send xConnect page view events to an external data store (perhaps via DEF?).

In doing so, we want to ensure that there's enough detail in the event information to be a self contained unit of value within the external data store. Let's take a look at a typical Page View Event in xConnect:

{
    "@odata.type": "#Sitecore.XConnect.Collection.Model.PageViewEvent",
    "CustomValues": [],
    "DefinitionId": "9326cb1e-cec8-48f2-9a3e-91c7dbb2166c",
    "ItemId": "163dcf8c-5191-4e77-90a1-babf352e7b1f",
    "Id": "3a2b4f6c-afc4-410b-9a6f-076636633d4f",
    "Timestamp": "2022-09-30T16:12:56.7227459Z",
    "Duration": "PT9.969S",
    "ItemLanguage": "en-US",
    "ItemVersion": 1,
    "Url": "/path-to-page",
    "SitecoreRenderingDevice": {
      "Id": "fe5d7fdf-89c0-4d99-9aa3-b5fbd009c9f3",
      "Name": "Default"
    }
  }

Looks like we're missing a full URL. How is an external system supposed to know that /path-to-page corresponds to https://www.yoursite.com/path-to-page? What if you have multiple websites on your Sitecore instance and you need to be able to distinguish page views between them? One way would be to add that information before you send it to the external system, or you could add the full URL in real time. Two potential approaches for adding the full URL in real time are explained below.

1. Override the Default Page View Event Creation Logic

This approach overrides the ConvertPageDataToPageViewEvent class to include the full URL instead of just the path. Once in place, this affects all future PageViewEvents.

I don't recommend this approach:

Whenever you override default behavior in Sitecore, there's always a risk of complications and unexpected consequences, so proceed with caution and test thoroughly.

Create a class that implements the ConvertToXConnectEventProcessorBase<PageData> interface and override as needed:

// ConvertPageDataToPageViewEvent.cs
using Sitecore.Analytics.Model;
using Sitecore.Framework.Conditions;
using SitecoreXConnect = Sitecore.XConnect;
using Sitecore.XConnect.Collection.Model;
using System;
using Sitecore.Analytics.XConnect.DataAccess.Pipelines.ConvertToXConnectEventPipeline;
using Sitecore.Web;

namespace Client.Foundation.DataExchange.Pipelines.ConvertToXConnectEventPipeline
{
  /// <summary>
  /// This class is automatically called when a session / interaction ends.
  /// </summary>
  public class ConvertPageDataToPageViewEvent : ConvertToXConnectEventProcessorBase<PageData>
  {
    protected override SitecoreXConnect.Event ConvertToEvent(PageData pageData)
    {
      Condition.Requires<DateTime>(pageData.DateTime, nameof(pageData)).IsUtc("{0}.DateTime should be of kind UTC");
      PageViewEvent pageViewEvent = CreatePageViewEvent(pageData);
      FillPageViewEvent(pageViewEvent, pageData);
      return (SitecoreXConnect.Event)pageViewEvent;
    }

    private PageViewEvent CreatePageViewEvent(PageData pageData)
    {
      PageViewEvent pageViewEvent = new PageViewEvent(pageData.DateTime, pageData.Item.Id, pageData.Item.Version, pageData.Item.Language);
      pageViewEvent.Duration = TimeSpan.FromMilliseconds((double)pageData.Duration);

      // Default setting of Url
      pageViewEvent.Url = pageData.Url.ToString();

      // Do what you need here
      // Note that there is no Sitecore context in this pipeline
      // The Sitecore.Web.WebUtil class is your friend
      pageViewEvent.Url = System.Web.HttpContext.Current.Request.Url.AbsoluteUri;

      // If you need to store arbitrary data, create a custom event instead (see the next section of this blog post)
      // In other words, don't do this:
      // pageViewEvent.CustomValues.Add("FullUrl", "Example Value");

      return pageViewEvent;
    }

    private void FillPageViewEvent(PageViewEvent pageViewEvent, PageData pageData)
    {
      if (pageData.SitecoreDevice == null)
        return;

      pageViewEvent.SitecoreRenderingDevice = new SitecoreXConnect.Collection.Model.SitecoreDeviceData(pageData.SitecoreDevice.Id, pageData.SitecoreDevice.Name);
    }
  }
}

<!-- Patch.config -->
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <convertToXConnectEvent>
        <processor type="Sitecore.Analytics.XConnect.DataAccess.Pipelines.ConvertToXConnectEventPipeline.ConvertPageDataToPageViewEvent, Sitecore.Analytics.XConnect">
            <patch:attribute name="type" value="Client.Foundation.DataExchange.Pipelines.ConvertToXConnectEventPipeline.ConvertPageDataToPageViewEvent, Client.Foundation.DataExchange" />
        </processor>
      </convertToXConnectEvent>
    </pipelines>
  </sitecore>
</configuration>

2. Create a Custom Event That Stores the Full URL

Before we continue, I recommend reading The Friendly Manual™️ on Sitecore events (yes, MORE reading!) because there are many ways you can veer away from best practices at this point; particularly around how you go about storing your custom data.

Sitecore Says:

Don't rely on the Data, DataKey, Text, and CustomValues properties on the Event class to store custom data. These properties are only available for legacy purposes.

You may feel an intense urge to forego Sitecore's advice.

Why Shouldn't I

After all, those legacy fields actually work just fine for storing data... But don't do it. It will likely complicate future upgrades and migrations. Instead, let's extend Sitecore like we're supposed to by creating a custom page view event.

Create a class that inherits ProcessItemProcessor. This will register the event data while the user is in session:

// RegisterCustomPageEvents.cs
using Sitecore.Analytics;
using System;
using Sitecore.Analytics.Pipelines.ProcessItem;
using Sitecore.Diagnostics;
using Sitecore.Analytics.Data;
using System.Collections.Generic;
using System.Linq;

namespace Client.Foundation.DataExchange.Pipelines.ProcessItem
{
  public class RegisterCustomPageEvents : ProcessItemProcessor
  {
    public override void Process(ProcessItemArgs args)
    {
      Assert.ArgumentNotNull((object)args, nameof(args));

      foreach (TrackingField trackingParameter in (IEnumerable<TrackingField>)args.TrackingParameters)
      {
        TrackingFieldProcessor.ProcessEvents(args.Interaction, trackingParameter);
      }  

      if (!Tracker.Enabled || Tracker.Current == null || !Tracker.Current.IsActive)
      {
        return;
      }

      RegisterCustomPageViewEvent(args);
    }

    private void RegisterCustomPageViewEvent(ProcessItemArgs args)
    {
      if (args.Item == null)
      {
        // Log or throw exception
        return;
      }

      // Create a corresponding item here: /sitecore/system/Settings/Analytics/Page Events/
      var customPageEventId = Guid.Parse("{7DDE228D-A30A-4670-A885-4EFB4054B561}");
      var ev = Tracker.MarketingDefinitions.PageEvents[customPageEventId];

      if (ev == null)
      {
        // Log or throw exception
        return;
      }

      var pageData = new PageEventData(ev.Alias, ev.Id);

      pageData.Text = "Custom Page View";

      var fullUrl = System.Web.HttpContext.Current.Request.Url.AbsoluteUri;

      pageData.DataKey = "Url";
      pageData.Data = fullUrl;
      pageData.ItemId = args.Item.ID.ToGuid();

      try
      {
        Assert.IsNotNull(Tracker.Current, "Tracker.Current");
        Assert.IsNotNull(Tracker.Current.Session, "Tracker.Current.Session");

        var interaction = Tracker.Current.Session.Interaction;

        Assert.IsNotNull(interaction, "Tracker.Current.Session.Interaction");
        Assert.IsNotNull(interaction.CurrentPage, "Tracker.Current.Session.Interaction.CurrentPage");

        // Prevent duplicates since this specific event should only fire once per page
        var doesEventExist = interaction.CurrentPage.PageEvents.Any(pe => pe.PageEventDefinitionId == customPageEventId);
        if (!doesEventExist)
        {
          Tracker.Current.CurrentPage.Register(pageData);
        }
      }
      catch (Exception ex)
      {
        Log.Error($"Unable to fire custom page event: {ex.Message}", ex, this);
      }
    }
  }
}

<!-- Patch.config -->
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <processItem>
        <processor type="Client.Foundation.DataExchange.Pipelines.ProcessItem.RegisterCustomPageEvents, Client.Foundation.DataExchange" />
      </processItem>
    </pipelines>
  </sitecore>
</configuration>

We've registered the event, but it won't actually get written to the database until we create a custom event converter which runs when the session ends.

Add a class that inherits ConvertPageEventDataToEventBase:

// ConvertPageEventDataToCustomPageViewEvent.cs
using Sitecore.Analytics.Model;
using Sitecore.Framework.Conditions;
using Sitecore.XConnect;
using System;
using Sitecore.Analytics.XConnect.DataAccess.Pipelines.ConvertToXConnectEventPipeline;

namespace Client.Foundation.DataExchange.Pipelines.ConvertToXConnectEventPipeline
{
  // https://doc.sitecore.com/xp/en/developers/93/sitecore-experience-platform/convert-a-page-event.html
  public class ConvertPageEventDataToCustomPageViewEvent : ConvertPageEventDataToEventBase
  {
    protected override bool CanProcessPageEventData(PageEventData pageEventData)
    {
      Condition.Requires(pageEventData, nameof(pageEventData)).IsNotNull();

      // Make sure the item you reference here is published and the marketing definitions are deployed
      if (pageEventData.PageEventDefinitionId == Guid.Parse("{7DDE228D-A30A-4670-A885-4EFB4054B561}"))
      {
        return true;
      }

      return false;
    }

    /// <summary>
    /// Create new custom event from pageEventData. All base Event properties are mapped automatically.
    /// </summary>
    /// <param name="pageEventData"></param>
    /// <returns></returns>
    protected override Event CreateEvent(PageEventData pageEventData)
    {
      // Creating a generic Event will work, but remember what Sitecore said...
      // Event customPageViewEvent = new Event(new Guid("7DDE228D-A30A-4670-A885-4EFB4054B561"), pageEventData.DateTime);
      // customPageViewEvent.ItemId = pageEventData.ItemId;
      // customPageViewEvent.DataKey = pageEventData.DataKey;
      // customPageViewEvent.Data = pageEventData.Data;

      // Custom model must be generated and deployed, else Sitecore will throw errors
      // https://doc.sitecore.com/xp/en/developers/93/sitecore-experience-platform/deploy-a-custom-model.html
      MyCustomPageViewEvent ev = new MyCustomPageViewEvent(pageEventData.DateTime, pageEventData.ItemId, pageEventData.DataKey, pageEventData.Data);

      return ev;
    }
  }
}

<!-- Patch.config -->
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <convertToXConnectEvent>
        <processor patch:after="processor[@type='Sitecore.Analytics.XConnect.DataAccess.Pipelines.ConvertToXConnectEventPipeline.ConvertPageEventDataToGoal, Sitecore.Analytics.XConnect']" type="Client.Foundation.DataExchange.Pipelines.ConvertToXConnectEventPipeline.ConvertPageEventDataToCustomPageViewEvent, Client.Foundation.DataExchange"/>
    </pipelines>
  </sitecore>
</configuration>

For reference, here is what your custom event type might look like:

// MyCustomPageViewEvent.cs
using System;
using Sitecore.XConnect;

namespace Client.Foundation.DataExchange.Models
{
  /// <summary>
  /// This event type is intended to be used as a data store which will ultimately be sent to and utilized by D365.
  /// This custom event type is required in order to store different data than what is stored by default.
  /// </summary>
  [Serializable]
  [FacetKey(DefaultFacetKey)]
  public class MyCustomPageViewEvent: Event
  {
    public const string DefaultFacetKey = "MyCustomPageViewEvent";
    public string FullUrl { get; set; }

    public MyCustomPageViewEvent(DateTime timestamp, Guid itemId, string dataKey, string data) : base(EventDefinitionId, timestamp)
    {
        ItemId = itemId;
        DataKey = dataKey;
        Data = data;
        FullUrl = data;
    }

    public static Guid EventDefinitionId { get; } = new Guid("7DDE228D-A30A-4670-A885-4EFB4054B561");
  }
}

For reference, here's a the default PageViewEvent that ships with Sitecore:

using System;

namespace Sitecore.XConnect.Collection.Model
{
  public class PageViewEvent : Event
  {
    public PageViewEvent(DateTime timestamp, Guid itemId, int itemVersion, string itemLanguage)
      : base(PageViewEvent.EventDefinitionId, timestamp)
    {
      this.ItemId = itemId;
      this.ItemVersion = itemVersion;
      this.ItemLanguage = itemLanguage;
    }

    public static Guid EventDefinitionId { get; } = new Guid("9326CB1E-CEC8-48F2-9A3E-91C7DBB2166C");

    public string ItemLanguage { get; set; }

    public int ItemVersion { get; set; }

    public string Url { get; set; }

    public SitecoreDeviceData SitecoreRenderingDevice { get; set; }
  }
}

In a future post, I'll be covering how you can selectively sync your custom events with DEF/D365 while filtering for specific event types / definition IDs.