While scholars and practitioners frequently laud the potential of networks to address complex policy problems, empirical evidence of the effectiveness of networks is scarce. This study examines how changes in network structure (centralization and transitivity), network composition (sector diversity and geographic range), and tie properties (stability and strength) influence community level outcomes. Relying on a statutory requirement in the state of Iowa requiring local governments to file all instances of intergovernmental and intersectoral collaboration, we measure collaboration networks in 81 counties over 17 years in the areas of crime and economic development. Using fixed effects models, we examine how changes in the structure and composition of these county‐level networks affect substantive policy outcomes. Our findings indicate that network properties matter, but that the specific properties may be context dependent. We find network centralization and stability are stronger predictors of crime while network composition is more strongly associated with economic development.