Extending DMS: Symposium Learnings, Part 2

In my last post, I discussed Sitecore's conception of DMS as a nexus for measuring, managing, and optimizing engagement value.  Today I'd like to share some concrete takeaways on how this can be done.  I attended three developer-track talks on DMS, and each reinforced my understanding of how this module can be extended by the developer to tie the customer experience to business value.

The DMS Data Model

Pieter Brinkman's talk, "Where is the data at? DMS data model explained" (slides, video) discussed how DMS translates the customer experience to in-memory structures and in database tables, and gave practical advice for managing the processing and storage demands of DMS.  My takeaways:

1. The talk reviewed the distinction between Visitors, Visits, and Pages.  Visitors, and stressed that the Visitor is cookie-based and so is defined at the browser level.  When you browse a website from home and from work, you are two different visitors from the point of view of DMS.  Each browser session is separate "Visit", and each page visited is recorded in Pages.  Pieter stressed that these are straightforward concepts, that can be explained easily to non-technical users.  Moreover, the design of the database is meant to support reporting and analysis, with visit information split into a number of ad hoc tables, such as "Browsers" and "OS", and "UserAgents," to simplify the task of report writing.
2. He discussed the importance of visitor Tags.  I was aware that DMS provided a mechanism for recording facts about a visitor in name-values pairs, but I was not aware of how this was wired into the functionality of Web Forms for Marketers and DMS Engagement Analytics reports.  When defining a field in a Web Forms for Marketers form, you have the option of associating it with a tag, so that the form can be prepopulated if the information is already known, and conversely, the tag can be set based on the users response.  Some tag names are embedded in existing Analytics reports; for example, First_Name and Last_Name in the Visitor Overview report.  Having a user submit a form with these fields will allow you to view his name on a Visitor Overview report, even if he has not created a user account.
3. He discussed several ways that DMS has been optimized for dealing with large volumes of data in a robust and efficient manner:
1. Visitor data is organized into visitor data sets, and then combined into a "commit data set," before being written to the database using a bulk insert operation. This minimizes the impact of I/O and transaction locking.
2. DMS can be configured to use separate servers for data capture, reporting, and automation, so that reporting operations do not impact data capture.  At a minimum, the DMS database should be housed on a different server from the main Sitecore databases, to avoid performance impacts due to resource contention.  
4. He recommended that thought be given to what data needs to be retained, and for how long.  DMS records a great deal of data, which can have negative impacts for reporting, maintenance, and backup time if it is not kept to a manageable size.

Extending the DMS Data Model

Ian Lowe's talk, "Integrating Google AdWords with Sitecore DMS" (slides, video), showed how this model could be extended to address a concrete reporting scenario, evaluating the effectiveness of Google AdWord campaigns.  He walked us through how Google structures AdWord data, and presented a module that downloads this data to DMS in a series of custom designed tables, so that it can be used in reports that combine DMS and Google data.  His example was from Sitecore's internal AdWord campaigns, and one point he stressed was that Sitecore uses Engagement Value internally with a great deal of rigor, with a point of value corresponding to a defined degree of likeliness for a future sale.  This allows them to do apples to apples comparisons of the costs of Google campaigns to the income they are expected to produce.   He also gave a number of practical tips for implementing this sort of project:

• Define your new tables with a common prefix so they are easy to identify.  He used "d2x", for "DMS to external data"
• Put your business logic in a SQL stored procedure. This makes it easy to adjust without doing a full deployment, which can be especially useful for tweaks based on business owner feedback.
• Use a Sheer interface to give your new functionality a polished native to Sitecore look.  The video link above gives a very useful walkthrough  of how to build a Sheer UI at the 34 minute mark.

Extending DMS Functionality

Lars Nielsen's talk, "Become the Marketer’s Best Friend – Using the DMS API to Extremes" (video), showed how a creative developer can extend the fundamental concepts of DMS personalization to meet a range of marketing scenarios:

1. Sitecore provides the ability to personalize based on search terms, but the out-of-the-box condition allows for comparing for a single value. Suppose you want to match any of a number of values, and suppose you want to use this list throughout the website? Lars walked us through the process of creating "Keyword bins", content items that stored comma-delimited lists of keywords, and developing a personalization condition that checks the search terms for a match against any of the keywords in a user-specified "bin."
2. Lars showed how to rank content based on visitor behavior. First, he gave some background on how DMS implicit personalization works, reviewing profiles, profile keys, and patterns. Profiles DMS provides the ability to profile content, and to evaluate patterns of visitor behavior. A "profile" defines a way of looking at a Sitecore item from a marketing perspective, such as indicating interest in a particular kind of product, or indicating a place in the sales cycle. To define a profile, you need to select a number of "profile keys," numeric values you can use to characterize content.  For example, on a travel site, you might define one profile for the regions of the world, and define a profile keys for each continent.  When a visitor to your website navigates through content, they accumulate profile scores based on their activity.  For example, a visitor who sees three pages that each have a Europe score of 5 and one page with an Asia score of 5 will accumulate 15 Europe points and 5 Asia points in his Region Profile.  Sitecore allows defining "Pattern" values that can be used in personalization rules.
   Lars explained that this matching is done by calculating the closest matching pattern in terms of Euclidean distance. So if one pattern is defined as Europe: 20, Asia: 0, and another is defined as Asia: 20, Europe:0, DMS will match the visitor's browsing profile to the Europe pattern, because (20-15)² is less than (20-5)². Rather than matching a visitor to a preset pattern, Lars showed that the same mathematics can be used to calculate the proximity of a visitors pattern to a series of content items. This allows you to sort content items based on which is likely to be of the most interest to a user, based on his or her browsing behavior.
   
3. The next scenario was using DMS to identify a visitor who has gone through a life changing event, such as getting married or having children. Using the assumption that such a change will be manifested by a marked change in browsing behavior from previous visits, and leveraging the fact that DMS treats visiting profiles as points in Euclidean space, Lars walked us through a condition that compared the angle of difference distance between the current visit and the users previous visits: "where the angle of current visit is X degrees different from the last Y visits." He pointed out that you had to measure the angle, rather than just the distance, because you want to distinguish a change in type of visit from merely continuing further on the same trajectory.
4. The next axiom of DMS that Lars messed with is that a Visitor in DMS terms is unique to a given browser, so that there is no connection in DMS terms between the you that browses on Chrome and the you that browses on an iPhone. This is because DMS is cookie based, and your identity as a visitor is determined by your Visitor cookie. Well, that's hardly ideal for scenarios where a visitor takes an action on the website that causes a link to be sent to his phone, certainly a common enough scenario. He showed how the Email Campaign Manager can be used to send a link that contained the Visitor cookie in the query string, and directing the user to a page that writes this cookie to the mobile device, deletes the current session, and does a redirect. Presto, the user's two devices now have the same visitor ID, so that personalization and reporting can be based on a complete view of the visitor.
   He mentioned in passing that DMS discards one-and-done sessions, assuming these are "bounces" or robot visits. This behavior is strikingly different from Google Analytics, which treats bounces as a key metric, and underscores DMS's focus on value over volume.
   
5. Finally, he pointed out that Engagement Value can come from sources outside of the website, such as activities like inbound phone calls that are visible to a CRM system. These can be introduced to the DMS database using AJAX services calls. These services can also be used to record Engagement Value for activities that are only visible client side.
   

An inspiring talk, rich in concepts, code, and advice.  My top take away is to view DMS functionality as a starting point, not an end point, for implementing business solutions.  You can do a lot with it!