From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale

Background¶

We are the Ad Platform team at Zalando Marketing Services (ZMS). We build the data pipelines that power Sponsored Products, the paid placements that advertisers use to promote their items across Zalando's catalog and product card pages.

When a customer sees or clicks a sponsored item, that interaction needs to be matched against the original auction that placed it. The auction record carries the ad server's delivery decision context; the interaction event carries the result. Joining the two is what turns a raw click into a billable charge and closes the feedback loop to our ad serving system.

This join has to happen in near-real time and within a bounded window. We limit it to 15 minutes: interactions that arrive after that are dropped. Missing an event means an advertiser action goes unbilled, so the match rate is a business-critical metric.