Optimizing Marketing Resource Allocation Through Reinforcement Learning

Abstract

Marketing managers allocate impressions, budget and offers across channels and creatives whose effectiveness is unknown in advance and varies across customers. The conventional approach, a fixed-horizon A/B test followed by full allocation to the winner, wastes impressions on inferior options during the test and ignores customer-level heterogeneity afterwards. This paper frames allocation as a sequential decision problem and evaluates bandit-based reinforcement learning (RL) policies against uniform allocation in a simulation of 20 campaigns, each serving 5,000 impressions across five creatives whose conversion probabilities depend on a three-dimensional customer context. Five policies are compared on cumulative regret, the expected conversions forgone relative to an oracle that always selects the best creative for each customer: uniform allocation, epsilon-greedy, UCB1, Thompson sampling and a linear contextual Thompson-sampling policy. The non-contextual learning policies reduce regret by 6–22% relative to uniform allocation, whereas the contextual policy reduces it by 76% with a fourfold lower variance across campaigns. The results show that most of the attainable gain comes not from learning which creative is best on average but from learning which creative is best for whom. The paper discusses implementation requirements, including context features, reward latency, exploration governance and the integration of bandit allocation with budget constraints, and positions contextual bandits as the practical entry point to RL in marketing.