Abstract: Arrhythmia is a common cardiovascular disease whose occurrence is mainly related to two factors: cardiac pacing and conduction. Some severe arrhythmias can even threaten human life. An electrocardiogram (ECG) records the changes in electrical activity generated during each cardiac cycle of the heart, which can reflect the human cardiac health status and help diagnose arrhythmias. However, because of the brevity of conventional ECGs, arrhythmias, which occasionally occur in daily life, cannot be detected easily. Automatic ECG analysis-based long-term heartbeat monitoring is of great significance for the effective detection of accidental arrhythmias and then for taking indispensable measures to prevent cardiovascular diseases in time. An ensemble extreme learning machine (ELM) approach for heartbeat classification that fuses handcrafted features and deep features was proposed. The manually extracted features clearly characterize the heartbeat signal, where RR interval features reflect the time-domain characteristic, and the wavelet coefficient features reflect the time–frequency characteristic. A 1D convolutional neural network (1D CNN) was designed to automatically extract deep features for heartbeat signals. These features were fused by an ELM for heartbeat classification. Because of the instability caused by the random assignment of ELM hidden layer parameters, the bagging ensemble strategy was introduced to integrate multiple ELMs to achieve stable classification performance and good generalization ability. The proposed approach was validated on the MIT-BIH arrhythmia public dataset. The classification accuracy reaches 99.02%, and the experimental results show that the performance of the proposed approach with fused features is better than those with only deep features and only handcrafted features.