Globally, building energy regulation has been an effective policy instrument for reducing energy use and carbon emissions. In Australia, the majority of regions address building performance through the National Construction Code. However, in 2004 the New South Wales government introduced a planning instrument called the ‘Building Sustainability Index’, known as BASIX. Until now there has been limited investigation of this sustainability index approach compared with addressing issues individually through building standards. This paper presents analysis of 94,648 building energy assessments in New South Wales and 190,286 from other Australian states to explore the impact of BASIX. The results show that the building code process delivers greater certainty and higher performance than through the sustainability index. The analysis shows that 58% of homes that pass BASIX would fail the National Construction Code for thermal comfort, and that the process of improving performance through the index is more cumbersome and slower than through individual building standards. Given the need to progressively increase house energy standards, governments should be seeking both certainty and higher performance outcomes. Failure to deliver community expected minimum performance risks locking in poor performance for long-life assets, and condemning a generation of households to unnecessarily higher energy bills.