To solve the homogeneous transformation equation of the form AX=XB in hand-eye calibration, where X represents an unknown transformation from the camera to the robot hand, and A and B denote the known movement transformations associated with the robot hand and the camera, respectively, this paper introduces a new linear decomposition algorithm which consists of singular value decomposition followed by the estimation of the optimal rotation matrix and the least squares equation to solve the rotation matrix of X. Without the requirements of traditional methods that A and B be rigid transformations with the same rotation angle, it enables the extension to non-rigid transformations for A and B. The details of our method are given, together with a short discussion of experimental results, showing that more precision and robustness can be achieved.