Session preview: Adobe's realtime identity graph with Flink

April 20, 2020 | by Fakrudeen Ali Ahmed

Are you ready for the Virtual Flink Forward this week? As a first time speaker, I am very excited to meet the Apache Flink community and share our experience using Flink at Adobe! This blog post gives a preview of my session “Real time identity graph” scheduled for April 22, 2020. 

If you haven’t done so already, go ahead and register for free to hear more about our use case with Apache Flink at Adobe and meet the Flink community. Here’s a preview of the session at Virtual Flink Forward this week:


Realtime identity graph

Adobe's identity graph links multiple identities for a user across multiple devices into a single unified identity. As a use case, this graph can be used to transfer a "build your own vehicle" session at a car website from desktop to mobile providing "personalization without a login". 


As an example, Alice has been planning to buy a car for some time and one afternoon during lunch break she goes to Ford’s website and spends 20-30 minutes customizing her car.

Adobe identify graph, Apache Flink, stream processing, big data, personalization example

Now she is all excited and goes to dinner with her husband in a restaurant and she wants to show her new car to her husband and discuss the buying decision.  She goes to the Ford site again on her phone this time. If it loads up like shown in red below [See figure] she is going to have to spend another 20 minutes getting back her favorite configuration and her partner may quickly lose interest. Instead, if it comes back by default in her favorite configuration as shown in green below [see figure] this is:

  1. A much more pleasant experience for users like Alice and her partner.

  2. She may decide to buy a Ford car because of this experience over a competitor.

Adobe’s identity graph enables this for users even if they don’t login in while maintaining their complete privacy and safety online.

Adobe, Identity Graph, Flink, stream processing, Apache Flink, personalization

This talk describes the design of Adobe’s  Real time identity graph on Flink which builds and updates this identity graph in near real time with 25B+ events a day, 2B+ devices and serving 500M+ users. It also covers how we arrived at Flink itself after evaluating multiple systems for our use case.


Our solution at a high level looks as follows:

Adobe, Apache Flink use case, Flink use case

Our scale is 25B+ events per day and this is one hour taken from our precision/recall studies with the old system we replaced.

Flink Application, Flink use case, Adobe

Evaluation of Frameworks

We objectively evaluated multiple streaming frameworks for our system and the summary of our results are:

Stream processing frameworks, evaluation, Flink, Apache Flink, Adobe, open source

Tune in to our talk to hear all the exciting details on this “beauty contest” for streaming frameworks!

In particular, our talk will cover:

  1. Adobe Experience Cloud, non-Oscar winning, non-photoshop part of Adobe which is a billion-dollar division in its own right. 

  2. Identity graph and its motivation.

  3. Objectively evaluating a software framework in particular in the context of streaming frameworks.

  4. Architecture of realtime identity graph on top of Flink, a complex system with 500M+ users and 25B+ events/day.

  5. Lessons learned in building this system in the context of Flink.



We turned a fuzzy decision of comparing multiple streaming frameworks into an objective decision-making methodology. For a sneak peek of our evaluation details take a look at the AdobeTech Blog. We used that decision to build a large scale event streaming system based on Flink successfully at a scale of 25B+ events per day. Tune in to our talk to listen more about this evaluation and our Flink-based system, realtime identity graph

Flink Forward, Registration, Virtual Event, Apache Flink, stream processingNew call-to-action




About the author: 

Fakrudeen Ali AhmedFakrudeen is an Architect in Digital Experience Cloud at Adobe with focus and expertise in Big data and ML technologies. Formerly, he was Senior Manager at Yahoo, managing Yahoo Front page, Yahoo Sports and Yahoo Finance content ranking and personalization system.

Fakrudeen is a member of Association for Computing Machinery (ACM) and American Association for the Advancement of Science (AAAS).

Topics: Flink Forward

Fakrudeen Ali Ahmed
Article by:

Fakrudeen Ali Ahmed

Related articles


Sign up for Monthly Blog Notifications

Please send me updates about products and services of Ververica via my e-mail address. Ververica will process my personal data in accordance with the Ververica Privacy Policy.

Our Latest Blogs

by Chen Qin September 21, 2021

The Apache Flink Story at Pinterest - Flink Forward Global 2021

On October 27, at the annual Apache Flink user conference, Flink Forward Global 2021, Pinterest Tech Lead, Chen Qin will deliver a keynote talk on “Sharing what we love: The Apache Flink Story at...

Read More
by Holger Temme August 16, 2021

Ververica named a 'Strong Performer' in Streaming Analytics by Forrester

We are excited to see Ververica Platform, developed by the original creators of Apache Flink, debut on the Forrester Wave™ 2021: Streaming Analytics report as a Strong Performer! Back in 2019,...

Read More
by Victor Xu July 13, 2021

Troubleshooting Apache Flink with Byteman


What would you do if you need to see more details of some Apache Flink application logic at runtime, but there's no logging in that code path? An option is modifying the Flink source...

Read More