There is a growing need for information about future climate conditions to support adaptation planning across a wide range of sectors and stakeholder communities. However, our principal tools for understanding future climate – global Earth system models – were not developed with these user needs in mind, nor have we developed transparent methods for evaluating and communicating the credibility of various climate information products with respect to the climate characteristics that matter most to decision-makers. Several recent community engagements have identified a need for “co-production” of knowledge among stakeholders and scientists. Here we highlight some of the barriers to communication and collaboration that must be overcome to improve the dialogue among researchers and climate adaptation practitioners in a meaningful way. Solutions to this challenge are two-fold: 1) new institutional arrangements and collaborative mechanisms designed to improve coordination and understanding among communities, and 2) a research agenda that explicitly incorporates stakeholder needs into model evaluation, development, and experimental design. We contrast the information content in global-scale model evaluation exercises with that required for in specific decision contexts, such as long-term agricultural management decisions. Finally, we present a vision for advancing the science of model evaluation in the context of predicting decision-relevant hydroclimate regime shifts in North America.