INTRODUCTION
What Works in Economic Development reviews evaluation literature relevant to economic development strategies. While economic development programs are difficult to evaluate, the literature reveals important information about the level of success achieved by various programs. It also demonstrates how positive results can be realized. A "what works" matrix follows the review of the evaluation studies. The matrix highlights the significant findings and key lessons that can be learned from this work.
In determining "what works" in economic development, a number of factors must be considered. First and foremost, what are the criteria by which economic development programs are to be judged? Is the goal to reveal whether the program has been implemented as intended or is the goal to assess the impact of the program?
An important distinction has been made between process evaluations and impact evaluations. Process evaluations focus on how a program is delivered; impact evaluations focus on program results (Bartik & Bingham 1997). The vast majority of economic development evaluations examine process. While they can be useful in helping to avoid problems in program implementation, they offer little in the way of measuring program success.
The tendency toward process evaluations largely springs from the difficulty of conducting impact evaluations. Establishing appropriate measures can be difficult in an evaluation, but may be particularly challenging when assessing the effectiveness of economic development programs.
To determine program effectiveness requires the ability to distinguish between changes due to the program and those due to factors unrelated to the program (Bartik and Bingham 1997). Establishing causal links is clearly a difficult task. It is also impossible to determine what might have occurred had the program not been initiated. Furthermore, it may be difficult to determine the appropriate time frame or geographic scope at which to assess the program.
The results of many economic development programs may not be immediately apparent. At what point should program effectiveness be assessed? Geography offers a similar dilemma. For example, should a work force development strategy be assessed according to its impact on regional labor force patterns or in terms of its ability to affect the distribution of employment within the region?
The goals of the program must always be kept in mind. According to Bartik and Bingham, "most economic development programs are so small relative to the community, any impact will be difficult to detect, but the impact may be large relative to the size of the program" (Bartik & Bingham 1997).
Some evaluators have focused on cost-benefit analyses to determine program effectiveness. Cost-benefit analysis is intended to show whether program benefits outweigh the costs of delivering the program. The advantage of this approach is that it addresses the issue of efficiency. The fact that program goals have been met does not necessarily mean the program was worthwhile.
Whether the project accomplished what leaders wanted it to accomplish, even if it did not make money (Pagano & Bowman), is often the measure of success. Cost benefit does not escape the problems of outcome evaluations, however. Evaluators must apply a monetary value to program benefits in order to conduct such an analysis. As if this were not a great enough challenge, determining program costs is also not a straightforward task. Cost to whom and benefit to whom?
In response to some of these issues, researchers have begun exploring the usefulness of new methods of evaluation being developed in other academic fields, such as the theory of change approach. The theory of change, which has attracted a great deal of attention, was first developed in an effort to establish better ways of assessing comprehensive community initiatives. It has been defined as "a systematic and cumulative study of the links between activities, outcomes and contexts of the initiative" (Connell & Kubish 1998).
This requires determining the intended outcomes, the activities implemented to achieve those outcomes, and the contextual factors that may have an effect on those activities and their ability to achieve the outcomes. The theory of change approach is appealing to some in the field of economic development because economic development programs share some of the same complexities as comprehensive community initiatives. While the approach does not totally eliminate the problems associated with establishing causal arguments, it is believed to reduce these problems by stating up front how activities are expected to lead to long-term outcomes.
Evaluation of economic development programs is still fairly limited. In addition to the methodological issues, political issues are involved. There is often fear that negative findings will have political consequences for those who have supported funding for the program. Economic development is practiced in a politically charged atmosphere with limited resources, a situation that can discourage honest evaluation.
When reviewing the findings of program evaluations, it is important to
keep in mind the issues described above. However, another caution is worthy
of mention. While program evaluation is important to understanding the
value of different economic development practices, it is essential to
remember the role of local context. What works in one area may not work
in another. Each economic region has its own opportunities and its own
constraints. This should always be kept in mind when assessing or developing
economic development strategies.