Help & Support

212.660.6600

Apache Unomi

A review of the Apache Unomi Customer Data Platform for use by marketing organizations.

Direct marketers face ever-increasing challenges identifying the needs and interests of customers in their target markets. While more information about people’s interests and purchasing habits is available than any other point in history, this data tends to be spread across multiple platforms and locations.

From a marketer’s perspective, it would make work considerably easier if this data were stored centrally. Customer Data Platforms (CDPs) have emerged to handle this problem. Data can be gathered, users can be tracked, and information can be stored in a common database or API. However, at the same time, there has been a rapid growth in legitimate concerns by the public about their privacy and how their information is being used.

New regulations, such as Europe’s GDPR, and several other laws in various states in the US are requiring that software give users the right to not be tracked, at least at an identifiable level. Recent rulings have required that individuals should be allowed to have their personal information be kept private.

Apache Unomi is an open source CDP designed to address both problems. It enables marketers to store information on customers from multiple locations in a central location while at the same time contains some built-in anonymizing functions to make it possible to protect consumers’ privacy. It becomes possible not only to make use of vast arrays of data, but also to make sure that data remains separate from the individuals from which it is gained.

Direct marketers face ever-increasing challenges identifying the needs and interests of customers in their target markets. While more information about people’s interests and purchasing habits is available than any other point in history, this data tends to be spread across multiple platforms and locations.

From a marketer’s perspective, it would make work considerably easier if this data were stored centrally. Customer Data Platforms (CDPs) have emerged to handle this problem. Data can be gathered, users can be tracked, and information can be stored in a common database or API. However, at the same time, there has been a rapid growth in legitimate concerns by the public about their privacy and how their information is being used.

New regulations, such as Europe’s GDPR, and several other laws in various states in the US are requiring that software give users the right to not be tracked, at least at an identifiable level. Recent rulings have required that individuals should be allowed to have their personal information be kept private.

Apache Unomi is an open source CDP designed to address both problems. It enables marketers to store information on customers from multiple locations in a central location while at the same time contains some built-in anonymizing functions to make it possible to protect consumers’ privacy. It becomes possible not only to make use of vast arrays of data, but also to make sure that data remains separate from the individuals from which it is gained.

  • Learn more about visitors.
  • Personalize their experiences by creating custom websites and mobile apps suited directly to the user
  • Comply fully GDPR and other privacy laws, enabling users to have control over their own analytics
  • Assign each user can to a personalized database, with its own rules, conditions, actions and more.

Features:

At a high level, Apache Unomi contains the following features:

  • User tracking
  • Event tracking
  • Goal tracking, scoring
  • Segmentation
  • Form input tracking
  • Download tracking
  • Personas
  • A/B testing
  • Privacy Management
  • Reporting
  • Profile management (includes visitors, contacts, leads, etc)

Use Cases

Let’s take a walk through Apache Unomi to see how it might handle a few use-case type scenarios.

The application can take inputs from a few user-end data sources. These could include visitors to a website, or it could be a mobile user using an application. These external inputs may feed information from the web browser or the mobile app to a content management system (CMS).

This system then gets the HTML and Javascript code and uses this to process the data, and send it all to a context server, which then loads the content into a context json file, which stores all of the information that contextualizes information about the user: who they are, are they the same entity, etc.

At this point, the server then sends back information to assist with the current request. This can include information such as past purchases, browsing history, etc. This information can be then sent back to the CMS. The process can be handled similarly with mobile apps: the data is sent from the app, to the CMS, to UNOMI which then feeds the data back, all the while interacting with the CMS or CRM software.

 

image1_10.png

 

In this way, a full profile of a user gets built, on the fly, and in real time. As a user interacts with a page, that data is then processed by Unomi to provide information to allow the CMS to deliver a customized web page or mobile app for the user.

Let’s break down the Unomi into components.

Profiles

Information about users is built dynamically. Starting with an individually known entity, each behavior and action is recorded into the profile object, which contains known information about who a user is and how they behave

Below is an example of a profile. This is a raw profile prior to any activities or actions occurring. It is a bare skeleton of a user profile.

profile = {
 "itemId":"10",
 "itemType":"profile",
 "version":None,
 "properties": {
  "firstName": "John",
  "lastName": "Smith"
},
 "systemProperties":{},
 "segments":[],
 "scores":{},
 "mergedWith":None,
 "consents":{}
}

Item

For any data being processed, we need to store the base information, including an identifying id and type of data. For instance, a social media item (a tweet) might look something like this.

{
 "itemId": "tweetNb",
 "itemType": "propertyType",
 "metadata": {
  "id": "tweetNb",
  "name": "tweetNb",
  "systemTags": ["social"]
},
 "target": "profiles",
 "type": "integer"
}

Event

Events are anything that occurs that can trigger various actions. An event could be an action on a web page or a mobile device, or it could be a weather event or anything else that might impact our data.

Here is a sample event structure

{
 "eventType": ,
 "scope": ,
 "source": ,
 "target": ,
 "properties":
}

To use a real world -type example, here’s an example page view event:

{
 "eventType": "view",
 "scope": "ACMESPACE",
 "source": {
  "itemType": "site",
  "scope": "ACMESPACE",
  "itemId": "c4761bbf-d85d-432b-8a94-37e866410375"
 },
 "target": {
  "itemType": "page",
  "scope": "ACMESPACE",
  "itemId": "b6acc7b3-6b9d-4a9f-af98-54800ec13a71",
  "properties": {
   "pageInfo": {
   "pageID": "b6acc7b3-6b9d-4a9f-af98-54800ec13a71",
   "pageName": "Home",
   "pagePath": "/sites/ACMESPACE/home",
   "destinationURL": "http://localhost:8080/sites/ACMESPACE/home.html",
   "referringURL": "http://localhost:8080/",
   "language": "en"
  },
  "category": {},
  "attributes": {}
  }
 }
}

 

Segments

Segments are used for grouping profiles together based on a series of conditions created when an action occurs. We can think of segments as something like target markets. For instance, if we divide users of a social media application as active and non-active, if the user performs any action at all, this person can be placed into a segment called “active users.”

Here’s an example of a sample segment (“leads”) as it is passed through the API and a set of Boolean conditions and sub-conditions that can help determine membership in a segment.

curl -X POST http://localhost:8181/cxs/segments \
--user karaf:karaf \
-H "Content-Type: application/json" \
-d @- <<'EOF'
{
 "metadata": {
  "id": "leads",
  "name": "Leads",
  "scope": "systemscope",
  "description": "You can customize the list below by editing the leads segment.",
  "readOnly":true
},
"condition": {
 "type": "booleanCondition",
 "parameterValues": {
  "operator" : "and",
  "subConditions": [
   {
   "type": "profilePropertyCondition",
   "parameterValues": {
    "propertyName": "properties.leadAssignedTo",
    "comparisonOperator": "exists"
   }
   }
  ]
 }
}
}
EOF

Conditions

Conditions are what they sound like; they are various identifiers or a list of parameter values for a specific condition. Conditions are set up as trees and are driven by Boolean expressions.

They can be simple, or they can have many complex determinants to identify very specific segments. Here’s a sample complex condition

{
 "condition": {
  "type": "booleanCondition",
  "parameterValues": {
   "operator":"or",
   "subConditions":[
   {
   "type": "eventTypeCondition",
   "parameterValues": {
    "eventTypeId": "sessionCreated"
   }
  },
  {
   "type": "eventTypeCondition",
   "parameterValues": {
    "eventTypeId": "sessionReassigned"
   }
  }
  ]
  }
 }
}

Privacy Features

As mentioned earlier, one of the unique features of Unomi is its ability to handle consumer privacy. It has a series of features to make it possible for applications to be fully GDPR compliant.

Apache Unomi contains the following privacy/identity management features

  • Profile Properties management
  • “personal identifier” properties
  • Endpoint anonymizing (which can erase the identifier above)
  • Consent management
  • Privacy management
  • Downloadable profiles for users

The consent management module is a separate API, which contains a scope, the type identifier for the consent, a status (grant, deny, revoke), the date, and the revocation date.

Example profile with a consent attached

{
 "profileId": "18afb5e3-48cf-4f8b-96c4-854cfaadf889",
 "sessionId": "1234",
 "profileProperties": null,
 "sessionProperties": null,
 "profileSegments": null,
 "filteringResults": null,
 "personalizations": null,
 "trackedConditions": [],
 "anonymousBrowsing": false,
 "consents": {
  "example/newsletter": {
   "scope": "example",
   "typeIdentifier": "newsletter",
   "status": "GRANTED",
    "statusDate": "2018-05-22T09:27:09Z",
   "revokeDate": "2020-05-21T09:27:09Z"
  }
 }
}

Rules

The rule engine operates in real time and can quickly generate this data at the moment a user performs an event, or an action occurs. Safety/privacy also works in real time. As it checks for information from trusted third parties, various rules become defined and are recorded in the user’s profile.

Actions occur when rules are satisfied, or conditions are met and will perform any create, read, update, or delete (CRUD) function defined.

The below diagram demonstrates how the process works. For example, we can retrieve data from a form event, copy it into the user profile, update profile segments, and send updated profile to a salesforce account.

 

image2_8.png

 

Data will merge in real time with other pre-existing data.

If we are to build this out, let’s take an example persona. Let’s say we want to identify the characteristics of a user that would make her a middle classed woman under the age of thirty.

We can create an example persona defined by income between 40K and 100K AND age <30

Let’s also say that she is an active user of a site and is interested in sports. This person could be identified as having read 10 or more pages with a tag labeled “sports” and has had at least 10 sessions within the last 10 days.

 

image3_8.png

 

Performance and Scalability

Apache Unomi is built for scalability. It is built on top of the Apache Karaf runtime environment, and uses ElasticSearch clustering. Unomi is easily extensible to help remove any bottlenecks caused during the data ingestion process or by rule processing. This can be handled by adding new nodes. The nature of the Karaf environment means that this will have little impact on processing speed.

Services

Apache Unomi provides the following services inside the Karaf cluster:

  • Shell commands
  • Segment services
  • Salesforce actions
  • Mailchimp actions
  • Weather
  • Rule engine
  • Profile service
  • Query service
  • Goals service
  • Personalization service
  • Event service
  • Segment (market) service
  • Definitions (rules, actions)

These are sent through the ElasticSearch Cluster.

 

Summary: Key takeaway

Apache Unomi can be an excellent tool for large enterprise-level organizations, or others who have a sufficiently capable IT staff. For direct marketing companies who wish to gain a true picture of their customers and target markets, Unomi allows consolidation of a tremendous amount of data which can be regularly updated in real time, and to be able to provide customized web pages and mobile sites to users.

One of its biggest strengths is that it takes into account growing public concerns (and laws) regarding privacy, and the right to both know what information is being collected about them, as well as the ability to opt-out. Apache Unomi is fully set up for privacy management and is GDPR compliant.

However, for smaller or organizations without technical ability, it is unfortunately a tool that might be considered out of reach, as it requires at least some basic developing skill to simply get it up and running. It is not a tool for business users.

Overall, it is powerful, flexible, fast, extensible and scalable, and takes into account security and safety of consumers. Developers can make excellent use of the ability to create a poweful API for managing and tracking users.

Integrations

Out of the box:

  • Salesforce
  • Twitter
  • Weather API
  • Mailchimp

As it uses a standard REST API and delivers results in JSON format, more extensions can easily be built

Xperra Star Ratings

Overall functionality useful to a direct marketer
4.5 /5

Apache Unomi is one of the most powerful tools of its type, and is excellent for consolidation of data into a central location. Data provided in real time can enable marketers to create customized and individualized responses and results to consumers based not only on demographic information, but also on actual behavior in real time.

One of Unomi’s biggest selling points for this purpose is the fact that it is open source, and therefore platform agnostic. A good CDP integrates data from multiple platforms and acts agnostically toward the source; it is meant as a clearing house, where one can draw clear pictures of customers and segments. For many for-profit vendors, there’s a strong tendency to invest resources into integrating their own products first, and to store the findings in a proprietary format, include licensing terms regarding data ownership, etc. This can have a negative impact on the overall quality of a CDP, and contrary to its goals.

In contrast, Apache Unomi treats all vendors the same, and makes it considerably easier to ensure that marketers can get a clear, real picture of who their clients are.

 

Intuitive User Experience
4 /5

Apache Unomi is, however, not designed for ease of use. It is first and foremost a platform for handling data, but it has no user interface. It is not a tool that one can simply open and start using. It is difficult to navigate, and information provided can be quite convoluted at first, until one spends a considerable amount of time organizing it.

That said, if one has development experience, it uses relatively easy to understand REST APIs which are in a standard JSON format, so after a little bit of time working with it, the workflow process can slowly become more streamlined.

Active Support Community
3 /5

Apache Unomi is still relatively new, so the community is not very large yet, but being part of the Apache universe means that there is a considerable amount of public support available

Apache Unomi Official Community

https://unomi.apache.org/community.html

They currently do not appear to have a bulletin board, however support is available in the form of two mailing lists, one for developers (https://lists.apache.org/[email protected]) and one for users (https://lists.apache.org/[email protected]) . There is also a Slack channel.

Support on Stackoverflow appears under the tag “Apache Karaf.”

Github: https://github.com/apache/unomi

Commits:

1799

Contributors:

23

Releases:

5

Watch:

14

Star:

68
Fork: 35

Commits/Contributors:

78

Minimal Technical Skill Required
1 /5

As mentioned earlier, Apache Unomi is not a tool for business users who do not also have developing skills. It is not commercial off-the-shelf software and has no graphical user interface; everything is handled via command-line.

Its purpose is specifically for developers who need the power that it can provide. It is essentially a “headless CDP” and is essentially a set of APIs. It serves as an alternative to building the CDP engine, or licensing another for software developers. This said, the functionality is very appealing for developers, as it has a wide array of features.

Related Experts

Data Architects

Data Architects

Project Manager

Project Manager

Database Administrator

Database Administrator

Data Analyst

Data Analyst

Related Solutions

Gain a 360⁰ View of Your Customers

Gain a 360⁰ View of Your Customers

Optimize Personalized Customer Journeys

Optimize Personalized Customer Journeys

Build a Single Customer View

Build a Single Customer View

Other Tools

Pimcore CDP
Customer Data Platform FREE Open Source

Pimcore CDP

The Pimcore Open Source Customer Data Platform (CDP) enables you to store and manage master data records of your customers. It offers impressive features for...

Pimcore eCommerce
ECommerce Platform FREE Open Source

Pimcore eCommerce

Pimcore is an Open Source enterprise application for outstanding B2B and B2C eCommerce customer experiences. Solve the challenges of omni-channel retail...

Mautic
Campaign Automation and Management Limited Open Source

Mautic

It's primary purpose is to track user behavior at an individual level. It allows the creation of a basic contact list, which can be entered manually or uploaded...