Abstract: Traditional cloud computing is developed from a high-performance cluster. Every server in the high-performance cluster has its own resources, including a CPU, memory, a network, I/O (Input/Output), a power system, and a heat dissipation system. Using software virtualization technologies such as the kernel-based virtual machine (KVM), Xen, VMware, and Hyper-V, these exclusive resources can be shared among these servers to improve the utilization rate. Although these technologies provide a great improvement in the resource utilization rate, some overhead in the process of software virtualization is inevitable. Server architecture and virtualization technology are the two factors that mainly affect cloud computing efficiency. With the rapid development of internet services, big data, and cloud computing, the cloud server has become mainstream instead of the traditional server. On the other hand, hardware virtualization technology has gradually developed. Compared with the traditional cloud computing solutions based on virtual machines, the cloud server based on hardware virtualization can achieve much higher efficiency to better meet cloud computing requirements by removing the software overhead. The cloud server’s design concept of configuration on demand, distributed sharing of hardware resource architecture, and construction method of hardware resource virtualization are presented. A three-level interconnection architecture of the cloud server is designed. In Level-1, the computing pool and the memory pool are built, while Level-2 is for the network pool, and Level-3 is for all resource pools. Different applications in these levels can be realized in the cloud server: Level-1 for computing-intensive applications, Level-2 for transactional applications, and Level-3 for virtual applications. A prototype system of a 16-processor cloud server using hardware virtualization architecture is designed and implemented. In this system, there are sixteen physical nodes. Every physical node is composed of a CPU and two DIMMs (dual inline memory modules). Different types of CPUs may be used in these physical nodes. Every four physical nodes form a computing module. In every computing module, a field-programmable gate array (FPGA)-based interconnection fabric controller (IFC) integrated network, storage, and general I/O resources is designed to interconnect these processors. All IFCs are interlinked. All the processors in this prototype system can share the network, storage, and general I/O resources to realize hardware resource virtualization through these IFCs. For the prototyping system, evaluation experiments on network performance tests by the Netperf program and storage performance tests by the FIO program are performed. The test results show that the prototyping system not only keeps the traditional cloud server’s advantages but also provides better scalability and performance. The advantages of this cloud server are in providing a high-density, high performance-to-cost ratio, a high performance-to-Watt ratio, and high scalability compared with the existing traditional cloud server.