Abstract: Due to the effect of the modulation part, the energy diffuses around the surrounding area of dominating frequencies and diminishes the readability of the time-frequency representation when short-time Fourier transform (STFT) is used to process nonstationary signals. However, when the instantaneous frequency slowly changes or is constant, the effect is small and can even be neglected. Thus, the time-frequency representations have high-energy concentration. Based on this feature, a novel method called iterative generalized short-time Fourier transform (IG-STFT) was proposed, which improved the readability of the time-frequency representation. First, the stationary mono-components are separated using iterative generalized demodulation. Then, the time-frequency representations of each mono-component are acquired using STFT. Finally, the time-frequency representation of the original signal is obtained according to the analysis results of STFT and the phase function. The analysis results of a planetary gearbox simulation signal and experimental signals verify the effectiveness of this method for analyzing nonstationary signals and diagnosing gear faults.