A novel method was designed to solve reinforcement learning problems with artificial potential field.Firstly a reinforcement learning problem was transferred to a path planning problem by using artificial potential field(APF),which was a very appropriate method to model a reinforcement learning problem.Secondly,a new APF algorithm was proposed to overcome the local minimum problem in the potential field methods with a virtual water-flow concept.The performance of this new method was tested by a gridworld problem named as key and door maze.The experimental results show that within 45 trials,good and deterministic policies are found in almost all simulations.In comparison with WIERING's HQ-learning system which needs 20 000 trials for stable solution,the proposed new method can obtain optimal and stable policy far more quickly than HQ-learning.Therefore,the new method is simple and effective to give an optimal solution to the reinforcement learning problem.
Estimating the interaction among neural networks is an interesting issue in neuroscience. Some methods have been proposed to estimate the coupling strength among neural networks; however, few estimations of the coupling direction (information flow) among neural networks have been attempted. It is known that Bayesian estimator is based on a priori knowledge and a probability of event occurrence. In this paper, a new method is proposed to estimate coupling directions among neural networks with conditional mutual information that is estimated by Bayesian estimation. First, this method is applied to analyze the simulated EEG series generated by a nonlinear lumped-parameter model. In comparison with the conditional mutual information with Shannon entropy, it is found that this method is more successful in estimating the coupling direction, and is insensitive to the length of EEG series. Therefore, this method is suitable to analyze a short time series in practice. Second, we demonstrate how this method can be applied to the analysis of human intracranial epileptic electroencephalogram (EEG) recordings, and to indicate the coupling directions among neural networks. Therefore, this method helps to elucidate the epileptic focus localization.
