The building energy efficiency labeling (BEEL) scheme has been adopted in China since 2008. However, until now, its effect on the actual building energy efficiency has not been accurately established. The objective of this study was to investigate this effect through a case study in Shanghai, China. Additionally, by performing a thorough review, potential barriers for implementing the BEEL scheme in major areas of China were analyzed. The results indicated that 1) the energy use intensity of labeled commercial office buildings in Shanghai followed a normal distribution, with an average value of 79.14 kWh/(m2a); 2) the energy-saving rate and energy consumption exhibited an inversely proportional relationship for residential buildings, but this relationship was not applicable for commercial office buildings; 3) for Shanghai, ground source heat pumps (GSHPs) and variable refrigerant volume systems are more appropriate for commercial office buildings, and GSHPs and split air conditioners are more useful for residential buildings. The potential barriers were classified into three main categories—technical, political, and awareness—and corresponding solutions were proposed. The authors hope that this study can promote the adoption of the BEEL scheme in China.