Low bulk diffusivity inside viscous semisolid atmospheric secondary organic aerosol (SOA) can prolong equilibration time scale, but its broader impacts on aerosol growth and size distribution dynamics are poorly understood. Here, we present quantitative insights into the effects of bulk diffusivity on the growth and evaporation kinetics of SOA formed under dry conditions from photooxidation of isoprene in the presence of a bimodal aerosol consisting of Aitken (ammonium sulfate) and accumulation (isoprene or α-pinene SOA) mode particles. Aerosol composition measurements and evaporation kinetics indicate that isoprene SOA is composed of several semivolatile organic compounds (SVOCs), with some reversibly reacting to form oligomers. Model analysis shows that liquid-like bulk diffusivities can be used to fit the observed evaporation kinetics of accumulation mode particles but fail to explain the growth kinetics of bimodal aerosol by significantly under-predicting the evolution of the Aitken mode. In contrast, the semisolid scenario successfully reproduces both evaporation and growth kinetics, with the interpretation that hindered partitioning of SVOCs into large viscous particles effectively promotes the growth of smaller particles that have shorter diffusion time scales. This effect has important implications for the growth of atmospheric ultrafine particles to climatically active sizes.