Denumerable semi-Markov decision chains with small interest rates

Rommert Dekker*, Arie Hordijk

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

4 Citations (Scopus)


In this paper we investigate denumerable state semi-Markov decision chains with small interest rates. We consider average and Blackwell optimality and allow for multiple closed sets and unbounded immediate rewards. Our analysis uses the existence of a Laurent series expansion for the total discounted rewards and the continuity of its terms. The assumptions are expressed in terms of a weighted supremum norm. Our method is based on an algebraic treatment of Laurent series; it constructs an appropriate linear space with a lexicographic ordering. Using two operators and a positiveness property we establish the existence of bounded solutions to optimality equations. The theory is illustrated with an example of a K-dimensional queueing system. This paper is strongly based on the work of Denardo [11] and Dekker and Hordijk [7].

Original languageEnglish
Pages (from-to)185-211
Number of pages27
JournalAnnals of Operations Research
Issue number1
Publication statusPublished - Dec 1991


Dive into the research topics of 'Denumerable semi-Markov decision chains with small interest rates'. Together they form a unique fingerprint.

Cite this