The ratio of plant carbon gain to water use, known as water use efficiency (WUE), has long been recognized as a key constraint on crop production and an important target for crop improvement. WUE is a physiologically and genetically complex trait that can be defined at a range of scales. Many component traits directly influence WUE, including photosynthesis, stomatal and mesophyll conductances, and canopy structure. Interactions of carbon and water relations with diverse aspects of the environment and crop development also modulate WUE. As a consequence, enhancing WUE by breeding or biotechnology has proven challenging but not impossible. This review aims to synthesize new knowledge of WUE arising from advances in phenotyping, modeling, physiology, genetics, and molecular biology in the context of classical theoretical principles. In addition, we discuss how rising atmospheric CO2 concentration has created and will continue to create opportunities for enhancing WUE by modifying the trade-off between photosynthesis and transpiration.