Deal Score: Unifying Deal Ranking in a Multi-Channel Auction Environment using MLPs

In many Demand-Side Platforms (DSPs), Private Marketplace (PMP) deals are integrated into the auction via a waterfall: preferred deals, private auctions, and open auctions are evaluated in a fixed, type-based order. This approach is simple to implement, but it fails to capture how external exchanges actually rank supply and often leads to suboptimal advertiser performance.

Symptoms included:

• Deal selection independent of expected surplus – The system did not choose bids (preferred deal vs private auction vs open auction) based on internal value that benefits the advertiser.
• Type-driven ranking – Deals were prioritized based on their type rather than realistic win probability given the auction dynamics.
• Exploitability of max-bid constraints – Advertisers could manipulate fixed priced deals to influence outcomes in ways that were misaligned with true value.

The underlying technical issue was that the legacy multi-tier algorithm could not compare deal types on a common scale. Without a unified scoring mechanism, the system could not rank all candidate routes by win probability or expected advertiser's value, which meant it could not consistently align internal decisions with external auction behavior.

Deal Score was designed to address this gap: a MLP NN model that uses supply and demand features and deal-specific win rate models to rank all deal types on a unified scale.

Legacy Waterfall: Where Heuristics Break Down

In adTech industry the deal waterfal is standard, the original waterfall algorithm treated each deal type as a separate “lane”:

1. Evaluate Preferred Deals (PD).
2. If no eligible PD is selected, evaluate Private Auctions (PA).
3. If no PA is selected, fall back to Open Auction (OA).

This produced several systemic problems:

• A high-value PA or OA bid could be dominated by a lower-value PD simply because PDs were evaluated earlier in the waterfall.
• Different components owned different parts of the pipeline, leading to inconsistent logic, overlapping experiments, and special-case filters.

Before proposing a replacement, we ran a full audit of the deal scoring and ranking pipeline. This surfaced:

• Legacy code whose effects were still active but poorly documented.
• Ad-hoc filters designed to “fix” specific deal problems rather than address the root ranking logic.
• Inconsistent handling of deals across components, making it difficult to reason about global optimality.

Once these issues were identified and cleaned up, we could treat the system as a more coherent optimization problem.

Formalizing the Problem: Expected Surplus and Win Rate

At a high level, each impression opportunity offers a set of candidate:

• One or more Preferred Deals (PD) – fixed price deals with access to premium inventory.
• One or more Private Auctions (PA) – deals with floors and a access to a lower tier premium inventory compared to PD.
• An Open Auction (OA) – the standard exchange auction.

For each candidate route $d$, we can think about:

• A bid $b_d$ (for PA/OA) or fixed price $p_d$ (for PD).
• A win probability $\Pr(\text{win} \mid d, b_d)$.
• An (implicit) notion of value from the advertiser’s perspective.

The expected surplus for a candidate can be written schematically as:

where InternalValue is a stand-in for how desirable that opportunity is given price and performance expectations. The exact decomposition can vary, but the core idea is:

For each impression opportunity, compute a scalar score that approximates “how good is it to choose this candidate for this impression?” and then pick the one with the highest score.

The legacy waterfall effectively ignored this formulation and instead relied on:

• Deal type ordering,
• Hard rules,
• And heuristics around floors.

Deal Score replaces that with a unified scoring function that is driven by deal-specific win rate models.

Step 1: Audit and Refactor the Deal Pipeline

Before implementing a new ranking mechanism, we:

1. Mapped the full deal lifecycle

   • From targeting and eligibility,
   • Through intermediate filters and experiments,
   • To final bid construction and response.

2. Cataloged all “special cases”

   • Experiment gates that altered ranking behavior.
   • Filters that removed deals based on static thresholds.
   • Different logic paths used for different deal types.

3. Removed or consolidated legacy hacks

   • Retired temporary fixes and code patches that had effectively become permanent.
   • Replaced scattered filters with a consistent, data-driven approach.

This refactor was necessary to ensure that:

• The ranking logic is deterministic and analyzable.
• Any improvements from Deal Score are not confounded by hidden rules in the legacy stack.

Step 2: Unified Deal Ranking via Win Rate Models

The core idea of Deal Score is to make win rate a first-class modeling target.

Win Rate Modeling

We train models that estimate:

where $d$ is the deal bid, $b_d$ is the bid (for PA/OA) or fixed price (for PD), and $x$ encodes contextual and deal-specific features, e.g.:

The model does not need to predict absolute win probabilities with perfect accuracy; it must produce well-calibrated, rank-consistent scores across deal types and contexts.

Deal Score Definition

For each candidate route $d$, we define a Deal Score that combines:

• Predicted win probability, and
• Deal-type specific pricing or value terms.

Conceptually:

• For Open Auction (OA) and Private Auctions (PA), ranking by Deal Score is approximately equivalent to ranking by effective price and win rate.
• For Preferred Deals (PD) with fixed prices, the model learns a probabilistic equivalent that allows them to be ranked on the same scale as auctions.

The system then selects the candidate $d^\*$ with the highest Deal Score:

This replaces the legacy order:

with a single-stage ranking problem.

Implementation Considerations

Several engineering choices were critical to making Deal Score practical and robust:

• Feature computation latency

  • Deal features and context signals must be computable in the real-time bidding path.
  • We reuse existing feature pipelines wherever possible to avoid duplication.

• Model serving and versioning

  • Models are deployed via a standard, real-time model serving framework.
  • Versioning and rollout are controlled through configuration and experiments.

• Monitoring and explainability

  • Per-deal and per-segment win rates are monitored before and after enabling Deal Score.
  • Deal Score outputs are logged alongside features to support offline analysis and debugging.
  • Interpretability tools help explain why one bid was chosen over another in specific cases.

• Fail-safe behavior

  • In the event of model unavailability or misconfiguration, the system can fall back to a safe, deterministic ranking (e.g., legacy waterfall).

Outcomes and Lessons Learned

Deal Score shifted deal selection from a type-based heuristic to a unified, model-driven ranking problem. Key learnings include:

• Unifying deal types on a common scoring scale is essential for optimizing PMP performance. Treating PD, PA, and OA as separate decision stages leads to structurally suboptimal outcomes.
• Win rate is a powerful principle: once you can estimate win probability by deal and context, you gain a natural axis for comparison and optimization.
• Cleaning up legacy logic is a prerequisite: without auditing and simplifying the pipeline, new models risk being obscured or contradicted by old rules.

By formalizing deal ranking as a data-driven scoring problem and grounding decisions in deal-specific win rate models, Deal Score provides a scalable foundation for improving PMP performance and aligning internal logic with the realities of modern, multi-channel auctions.