The Edna McConnell Clark Foundation believes that a well-conducted, rigorous, independent evaluation is the best way to provide persuasive empirical evidence that a program is effective. Our experience suggests that the earlier an organization embraces performance measurement and evaluation, the sooner it will be prepared and likely to grow. Especially given the severe constraints on public funds and the keen competition for them, government would be doing a disservice to taxpayers and to the vulnerable populations social programs are intended to assist if it did not require thorough, rigorous evaluations before investing large amounts of money in expanding such programs on a significant scale. Consequently EMCF has made major investments in helping grantees build their evidentiary base, including rigorous independent evaluations of their programs. 

In an ideal world, an experimental evaluation or randomized controlled trial (RCT), which compares participants in a program to a randomly assigned control group of peers who do not participate, provides the highest standard of proof and level of confidence that a program works.

In the real world, an RCT may not always be feasible, for several reasons:

• A program may not be mature enough operationally or an organization’s performance management and measurement systems insufficiently developed to evaluate impact.
• A program may not be large enough to produce participants in sufficient numbers and/or a control group of sufficient size to yield a reliable measure of effectiveness.
• In some instances it may be impractical or unethical to use random assignment to create and differentiate a pool of participants and a control group.

Under some of the above circumstances, a third-party, quasi-experimental evaluation comparing participants to a comparison group that is not randomly assigned may represent the highest proof point a program is capable of reaching at that point in time. If an EMCF grantee is at an early stage of organizational development or operating on a small or medium scale and an RCT is therefore premature, our goal is to advance the program with capacity-building investments to the level where an RCT can be conducted.

Strong impact data is especially critical for programs that serve disadvantaged young people, since until recently there have been few examples of reliably evaluated successful programs. Too often, philanthropists, policymakers and program administrators rely on outcome data alone to assess effectiveness. Outcome data is invaluable when monitoring the implementation of a program, but many young people, as they mature, are able to find their way even without program intervention. The challenge for evaluators and program administrators alike is to learn what net difference programs make above and beyond what young people would have done on their own or as a result of the existing infrastructure of options available to them.

A well-designed, rigorous, independent evaluation benefits an organization in other important ways in addition to assessing a program externally. The data collection and analysis an evaluation requires provide an opportunity and means to improve a program internally. The information it gleans helps an organization determine what works and what does not, and adjust its program(s) accordingly. Preparing for and undergoing an evaluation has prompted several of our grantees to design or improve performance measurement systems and to use these as management tools. The discipline of evaluation, understanding a program’s strengths, limitations, and areas needing improvement, makes an organization more realistic, sophisticated and effective—and better prepared to achieve sustainable growth and serve greater numbers of low-income youth.

Evaluation is an ongoing, dynamic activity, and even after an organization has proven the impact of its program(s) with an RCT, it must continue to test itself and demonstrate that it is still replicating the program(s) with fidelity to the model. For an example of how one EMCF grantee, Nurse-Family Partnership, has used evidence and evaluation to demonstrate impact, replicability and cost savings, see their website page on Evidence-based policy.