The problems of robust exponential stability in mean square and delayed state feedback stabilization for uncertain stochastic systems with time-varying delay are studied. By using Jensen's integral inequality and combining with the free weighting matrix approach, new delay-dependent stability conditions and delayed state feedback stabilization criteria are obtained in terms of linear matrix inequalities. Meanwhile, the proposed delayed state feedback stabilization criteria are more convenient in application than the existing ones since fewer tuning parameters are involved. Numerical examples are given to illustrate the effectiveness of the proposed methods.