Indicators show or are measures that show progress toward or achievement of outcomes. They are observable, measurable evidence of change. Social indicators help you figure out if your actions are having a positive effect on people's lives.Therefore, it is useful to identify indicators in order to know what you want to measure and what data to collect. Because indicators are defined in relation to desired outcomes, you may wish to explore Typical Social and Civic Outcomes first. This section includes:
Characteristics of Indicators
Indicators are specific measurable changes that can be seen, heard or read to demonstrate that an outcome is being met. Indicators should be easily measurable (within reason) and meaningful to stakeholders. In selecting indicators five factors may be influential:
- What — the condition, behavior, or characteristic to be measured
- Who — the target population to be measured
- How much — the degree of change that is expected
- How many — the amount of change among the target population that would indicate a successful level of achievement
- When — the time frame in which this change should occur
Indicators reflect the change that results during a program or after it has been launched. The W.K. Kellogg Foundation distinguishes between leading indicators (those that signal movement in an intended direction, versus lagging indicators (the ripple effects of what is taking place). Given the scope and time involved in collecting and analyzing data, it is important to limit indicators to those that are most important to measure.
Measuring Indicators
How do you measure indicators? Your approach to collecting data can be quantitative or qualitative or both.
Quantitative methods measure changes expressed as numbers or statistics, e.g. frequency of behaviors, number of attendees. Quantitative data can also measure changes in attitudes, beliefs, and perceptions when respondents express opinions in the form of ratings or rankings. An advantage of a quantitative data collection approach is that data can be easier to collect and analyze than qualitative information. However, data that is easy to gather may not be the most illuminating or insightful.
Qualitative data relies on observations of changes with the intent of understanding patterns and relationships. Data are often in the form of descriptions, narratives, and open-ended responses. Qualitative data might be used to capture individual and collective stories of social change, including personal perceptions. Sometimes qualitative evidence is the only way to document changes in relationships and contexts.
To measure evidence of change in community members’ perceptions of immigrants, for example, an indicator of change might be increased positive attitudes about immigrants as found in community members’ actual expressed sentiments and stories (gathered via interviews, focus groups, ethnographic process, or creative story circles). Qualitative data can be systematically collected and then categorized for analysis to reveal changes that may seem hard to measure.
Perhaps the most robust type of indicator allows for both types of data collection approaches. For example, to measure increased positive attitudes about immigrants, surveys might be administered periodically in addition to using the methods described above.
Tips on Selecting Indicators
Because perceptions of a program’s success are largely understood by what is measured, indicators should be selected carefully. Consider the following:
- Involve stakeholders in defining indicators. Terms such as “success” and “progress” can be interpreted in a myriad of ways. Understanding what partners and various stakeholders value most, how they interpret success, and what they will view as evidence of success is important in selecting indicators.
- Consider both quantitative and qualitative indicators. What is most important to a program or initiative may not be easiest thing to measure. In collecting data that is easily accessible or purely quantitative, stakeholders run the risk of being reductive in assessing the true and deeper impact of a program. Conversely, a cursory anecdotal review of a program done by those who are predisposed to view it as successful would not likely hold up to scrutiny from outsiders. Michael Quinn Patton refers to this as “statistics versus meaning. He asserts that statistics tell you how many but they don’t tell you why.
- Determine if your outcomes—and therefore the indicators you define—are short-, intermediate- or long-term. Measurement of short-term change will most likely focus on the implementation of a project (How did it work? How effective was the design?) or on establishing a baseline from which to assess change down the road. Intermediate change can be measured by defining benchmarks. A benchmark is a type of indicator that refers to the level of change an organization expects to make from its baseline. Most evaluators and researchers agree that social impact takes time to occur and that outcomes are only measurable in the intermediate- or long-term. The longer a program runs, the more complex its selection of indicators and the more likely, or even necessary, it becomes to use benchmarks.
- Consider a program’s context and its complexity as this will affect results and the selection of indicators. In complex systems, change is assumed not to be as purely linear as a logic model or outcomes chain implies. Rather, progress or change jumps forward, stalls, circles around, or even doubles back on itself, based on a variety of factors. Indicator selection, therefore, needs to take into account this dynamic of flux (search developmental evaluation and complexity references on this site). You will want to select indicators at the beginning of a program or project in order to plan for data collection, but also allow for indicators to be defined or revised as new outcomes emerge during program implementation.
- Decide on a modest range of realistic indicators. It is nearly impossible for a few indicators to capture complex social change, particularly since change tends to happen incrementally and can be attributed to numerous inputs. On the other hand, having too many outcomes and indicators can aim data collection in many different directions, making it difficult to gather credible evidence.
- Avoid common pitfalls. One frequent mistake is that people choose outcomes that they believe they can easily achieve and measure based on the desire to show success. Conversely, they may ignore outcomes that are difficult to achieve and measure. Another mistake is to disregard negative outcomes or not be alert to unexpected outcomes that, if recorded and assessed, can be just as important in terms of understanding impact. Yet another pitfall is the tendency to attribute causality for social change to one modest program when that change is caused by multiple factors.