This paper considers a first passage model for discounted semi-Markov decision processes with denumerable states and nonnegative costs. The criterion to be optimized is the expected discounted cost incurred during a first passage time to a given target set. We first construct a semi-Markov decision process under a given semi-Markov decision kernel and a policy. Then, we prove that the value function satisfies the optimality equation and there exists an optimal （or ε-optimal） stationary policy under suitable conditions by using a minimum nonnegative solution approach. Further we give some properties of optimal policies. In addition, a value iteration algorithm for computing the value function and optimal policies is developed and an example is given. Finally, it is showed that our model is an extension of the first passage models for both discrete-time and continuous-time Markov decision processes.
Let {Xn,n ≥ 1} be a sequence of identically distributed ρ^--mixing random variables and set Sn =∑i^n=1 Xi,n ≥ 1,the suffcient and necessary conditions for the existence of moments of supn≥1 ｜Sn/n^1/r｜^p（0 〈 r 〈 2,p 〉 0） are given,which are the same as that in the independent case.
