Morphing for Consumer Dynamics: Bandits Meet Hidden Markov Models

Gui Liberali*, Alina Ferecatu

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

4 Citations (Scopus)
127 Downloads (Pure)


Websites are created to help visitors take an action, such as making a purchase or a donation. As visitors browse various web pages, they may take rapid steps toward the action or may bounce away. Websites that can adapt to match such consumer dynamics perform better. However, assessing visitors’ changing distance to the action, at each click, and adapting to it in real time is challenging because of the sheer number of design elements that are found in websites, that combine exponentially. We solve this problem by matching latent states to web page designs, combining recent advances in multiarmed bandit (MAB), website morphing, and hidden Markov models (HMM) literature. We develop a novel dynamic program to explicitly model the trade-off firms face between nudging a visitor to later states along the funnel, and maximizing immediate reward given current estimates of purchase probabilities. We use an HMM to assess visitors’ states in real time, and couple it with an MAB model to learn the effectiveness of each design × state combination. We provide a proof of concept in two applications. First, we conduct a field study on the Master of Business Administration website of a major university. Second, we implement our algorithm on a cloud server and test it on an experimental online store.

Original languageEnglish
Pages (from-to)341-366
Number of pages26
JournalMarketing Science
Issue number4
Publication statusPublished - 10 Feb 2022

Bibliographical note

Funding Information:
History: Olivier Toubia served as the senior editor and Sha Yang served as associate editor for this article. Funding: This work was supported by the Erasmus Research Institute of Management, the Erasmus Corporate Marketing and Communication, the Erasmus Centre for Optimization of Online Experi-ments (, HyperMorphing Technologies, and Supplemental Material: Data and the online appendices are available at 2021.1346.

Publisher Copyright:
© 2022 INFORMS.

Research programs



Dive into the research topics of 'Morphing for Consumer Dynamics: Bandits Meet Hidden Markov Models'. Together they form a unique fingerprint.

Cite this