A mathematical approach for evaluating Markov models in continuous time without discrete-event simulation

Joost van Rosmalen*, Mehlika Toy, James F O'Mahony

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

14 Citations (Scopus)


Markov models are a simple and powerful tool for analyzing the health and economic effects of health care interventions. These models are usually evaluated in discrete time using cohort analysis. The use of discrete time assumes that changes in health states occur only at the end of a cycle period. Discrete-time Markov models only approximate the process of disease progression, as clinical events typically occur in continuous time. The approximation can yield biased cost-effectiveness estimates for Markov models with long cycle periods and if no half-cycle correction is made. The purpose of this article is to present an overview of methods for evaluating Markov models in continuous time. These methods use mathematical results from stochastic process theory and control theory. The methods are illustrated using an applied example on the cost-effectiveness of antiviral therapy for chronic hepatitis B. The main result is a mathematical solution for the expected time spent in each state in a continuous-time Markov model. It is shown how this solution can account for age-dependent transition rates and discounting of costs and health effects, and how the concept of tunnel states can be used to account for transition rates that depend on the time spent in a state. The applied example shows that the continuous-time model yields more accurate results than the discrete-time model but does not require much computation time and is easily implemented. In conclusion, continuous-time Markov models are a feasible alternative to cohort analysis and can offer several theoretical and practical advantages.

Original languageEnglish
Pages (from-to)767-779
Number of pages13
JournalMedical Decision Making
Issue number6
Publication statusE-pub ahead of print - 28 May 2013

Research programs

  • EMC NIHES-01-66-01
  • EMC NIHES-02-65-01


Dive into the research topics of 'A mathematical approach for evaluating Markov models in continuous time without discrete-event simulation'. Together they form a unique fingerprint.

Cite this