The current policy environment has created a historic opportunity to improve healthcare delivery in Pennsylvania through the effective use of community health workers (CHWs). The long history and rapidly expanding evidence-base suggest that CHW programs have potential to improve health outcomes among disadvantaged populations. Yet, history reveals that there is great variability in the quality of CHW programs. CHW programs that are poorly designed or implemented can lead to wasted resources and potentially adverse patient outcomes. As Pennsylvania considers policies for training and certifying Community Health Workers, the state should consider optimal measures for evaluating effectiveness.