Underdetermined blind signal separation (BSS) (with fewer observed mixtures than sources) is discussed. A novel searching-and-averaging method in time domain (SAMTD) is proposed. It can solve a kind of problems that are very hard to solve by using sparse representation in frequency domain. Bypassing the disadvantages of traditional clustering (e.g., K-means or potential-function clustering), the durative- sparsity of a speech signal in time domain is used. To recover the mixing matrix, our method deletes those samples, which are not in the same or inverse direction of the basis vectors. To recover the sources, an improved geometric approach to overcomplete ICA (Independent Component Analysis) is presented. Several speech signal experiments demonstrate the good performance of the proposed method.