High resolution climate simulations have the potential to outperform standard simulations in certain aspects but also face a number of challenges. These challenges include the increased computational expense involved in both model spinup and tuning, and a general lack of multi-member ensembles for assessing forced responses and internal variability. In this work, the performance of several models is benchmarked across a range of resolutions using NCAR's Climate Variability Diagnostics Package and Climate Model Analysis Tool. Where possible, multiple realizations are assessed to gauge the intrinsic noise of benchmark metrics and the significance of inter-model differences. The results presented highlight the risks and tradeoffs involved in high resolution modeling and the challenges involved in demonstrating fitness for purpose of high resolution climate predictions.