A new algorithm is proposed, which immolates the optimality of control policies potentially to obtain the robusticity of solutions. The robusticity of solutions maybe becomes a very important property for a learning system when there exists non-matching between theory models and practical physical system, or the practical system is not static, or the availability of a control action changes along with the variety of time. The main contribution is that a set of approximation algorithms and their convergence results are given. A generalized average operator instead of the general optimal operator max (or min) is applied to study a class of important learning algorithms, dynamic programming algorithms, and discuss their convergences from theoretic point of view. The purpose for this research is to improve the robusticity of reinforcement learning algorithms theoretically.
Mobile ad hoc networks (MANETs) have become a hot issue in the area of wireless networks for their non-infrastructure and mobile features. In this paper, a MANET is modeled so that the length of each link in the network is considered as a birth-death process and the space is reused for n times in the flooding process, which is named as an n-spatial reuse birth-death model (n-SRBDM). We analyze the performance of the network under the dynamic source routing protocol (DSR) which is a famous reactive routing protocol. Some performance parameters of the route discovery are studied such as the probability distribution and the expectation of the flooding distance, the probability that a route is discovered by a query packet with a hop limit, the probability that a request packet finds a r-time-valid route or a symmetric-valid route, and the average time needed to discover a valid route. For the route maintenance, some parameters are introduced and studied such as the average frequency of route recovery and the average time of a route to be valid. We compare the two models with spatial reuse and without spatial reuse by evaluating these parameters. It is shown that the spatial reuse model is much more effective in routing.
