The Animating Democracy site is currently being upgraded. During this time, new visitors will not be able to create a new account and current users will not be able to add new content. Apologies for any inconvienence.
Stories & Examples
On-the-ground examples of evaluation in action!
These case studies, evaluation reports, and profiles offer insights that may help you:
- define civic/social outcomes and indicators
- hone the purpose of or focus your evaluation
- make choices regarding approach and methods
- find creative ways to report findings.
Check out:
In-Depth Case Studies from Animating Democracy's Arts and Civic Engagement Impact Initiative.
Add your own:
Evaluation case studies, reports or stories. Share your work by contacting animatingdemocracy@artsusa.org
This item relates to the Preliminary Menu of Possible Outcomes/Indicators/Measures from the Starksboro Art and Soul Project, as both are part of the Art and Soul Project. This item is a ten page community survey with thirty questions. The survey is an adaptation of a standard survey developed by the Orton Family Foundation to use in tracking changes in perceptions about land use planning in all their Heart and Soul community projects. The Preliminary Menu is a table relating outcomes, indicators, and data collection methods for the Art and Soul Project.)
The the final portion of the survey relates specifically to the Art and Soul Project: it asks how much people know about the project, what their vision and goals are, and how involved they may wish to be. Visually pleasing but rather lengthy, this is a nice sample instrument. Complete information about the project in which the survey was used is found at: http://www.orton.org/projects/starksboro. The first questions are basic biographical/demographic items. The next four questions relate to land use perceptions and participants' willingness to get involved in the issue. The next three questions ask participants to check off boxes relating to their how much information they take in about community planning and how they communicate their concerns. Three questions are about participants' perceptions of how well the community and its leadership are doing with the issues at hand.
Conflicts in Iraq and Afghanistan have produced a wave of returning veterans suffering from both physical and emotional traumas as well as families, communities, and a society in need of ways to understand, adjust, and heal. Writer and “former military kid” Maranatha Bivens characterizes ways that art is raising awareness of the issues facing service members, bridging gaps in knowledge and communication between veterans and civilians, and offering veterans paths to healing and reintegration in family and community life.
Artists are creating work that enriches the public narrative about service, military culture, and the politics and effects of war through commentary, bearing witness, and protest. A breadth of artistic engagement at the personal and organizational level includes: documentary and classical theater, original artworks made from the pulp of old uniforms, and documentation of war zone graffiti, to name just a few examples. Some of the most compelling work is led and created by veterans themselves. Arts and creativity are being applied significantly as tools for health and well being. The National Initiative for Arts and Health in the Military brings together the military branches with civilian agencies and nonprofits to advance policy and practice toward integrating arts in holistic treatment. On the ground, creative arts therapists, artists, and arts organizations work across the military lifecycle, providing therapeutic and educational services. Finally, the paper reflects on how the arts may be applied as a tool for diplomacy to build transnational community connections, bridge cultural distinctions, and contribute to economic vitality.
In order to better understand film as an agent of social change, this article offers examples of documentary films that have led to change in viewer behavior, public policy, and discourse. After examining the successful impact of these examples, the authors offer planning tools for outreach campaigns, an approach to assessing impact, and lessons learned. This article may be useful to the strategic design of programs and outreach efforts that aim to affect change.
Although documentary films reach fewer people than their mass media counterparts, they still have the power to inspire action and ignite social change, especially when the film making process itself is considered as a change agent. Technology, community engagement activities, and outreach campaigns add to a film’s ability to make an impact on its audiences. The authors analyze three documentary films that possess the components of successful outreach and engagement campaigns. These components include: clear goals, a specific plan, an appropriate level of partnership, sufficient expertise and resources, a defined timeline, and a plan to track impact. In Figure 1, these components are placed on a continuum ranging from simple promotion to more complex community engagement.
Moving from an intuition to more concrete measures that show the impact of film and media on social change, the authors note that it is important to “look at a variety of data sources such as survey results, case studies, and anecdotal data.” They suggest using website technology to track output (number of screenings) and outcome (results of those screenings) measures. Figure 2 shows the Fledgling Fund’s “Dimensions of Impact” framework that the organization uses to assess its funded projects. The authors discuss the importance of each dimension and offer sample measures of impact (e.g., audience size, blog participation, mention in policy discussion, etc).
Battery Dance employed a pre and post program participant survey to assess its program, Dancing to Connect - Iraq. In doing so, the dance company and the program participants explored areas of self-expression, attitude and perception changes, and conflict resolution. Three attached documents provide more detail of these evaluation tools: 1) a final program report, 2) an executive summary of an evaluative report with survey questions and results, and a graphical representation of survey results.
Working collaboratively with American and local partners, Battery Dance Company of New York used the art of dance as a medium for healing and youth empowerment in Iraq. The program was sparked by a Marine returning to Iraq as part of a cultural diplomacy program. Employing Battery Dance Company’s signature arts education methodology, Dancing to Connect (DtC), a pair of American teaching artists engaged with 28 Iraqi students, ages 17-22, and two local teacher trainees over a period of one week. They worked in a group that crossed gender, religious, social and geographic boundaries. Together and under the guidance of BDC’s teaching artists, the students created choreography that spoke to the issues of inclusion/exclusion, the struggles of living in a war-torn country, and their hopes for a better future. As the process went forward, the initial differences between the students melted away.
Over the course of the week, the students learned the craft of choreography as a vehicle for expressing their emotions and creative visions. None of the participants had taken formal dance classes in the past. Many were students at the Institutes of Performing Arts where they were studying acting or directing. As such, everyone began the process on a similar, very fundamental level. The differences between the students from the two cities were noticeable at the beginning of the workshop. They ate lunch on different sides of the courtyard and expressed passionate opinions and observations about one another in discussions. As the workshop developed, and the teaching artists initiated the mixing of demographics, the differences disappeared. On the last day of the workshop the students from both cities and religions were intermingled, professing friendship, and singing traditional songs together.
The attached evaluation tools and reports show the company's efforts in assessing the impact of this work on participants' lives and attitudes.
"Connecting Californians" reports on a research project completed in 2000, that explored story as a powerful means of building community. The project conducted a search in each of California's 58 counties to find projects that engaged residents in a public performance or story about local history and life. Maps were created to represent the various projects. It is a helpful model for collaborative planning and discussion regarding the arts, culture, and civil society. Although it does not offer frameworks or tools, it is a useful report sample and resource for arts and civic engagement practitioners.
Over the course of ten months, the research team interviewed more than 100 practitioners, educators and policy makers with experience in the arts, humanities and civic culture. They interviewed and convened repesentatives of projects at the intersection of the arts, the humanities, grassroots narrative and community organizing to explore the role that a cultural organizer plays in promoting and sustaining collaborations, facilitating relationships and encouraging local leaders to emerge. Participants included artists, humanists, organizers and residents and representatives of their civic and religious organizations. This report presents the findings of that inquiry and questions for the future.
A section on program design discusses three core values — engagement, inclusion and inquiry. The report includes six comprehensive appendices. Appendix A: Three Local Projects presents brief case studies of public performances springing from the history, aesthetics and issues in three California communities. Appendix B: Evidence of Public Performance Based on Local Life offers a scan of the field. Appendix C gives information on two focus groups. Artists, organizers, educators and funders that came together to discuss the role of story in building community. Appendix D reports on national and California interviews. Appendix E includes three research essays. Writer and critic Linda Frye Burnham interviewed practitioners and thinkers in the field grappling with issues at the intersection of community organizing, art and the humanities, producing three monographs. Appendix F is a literature review [Adapted from the Community Arts Network website.]
The strength of Counting New Beans is in its impressive list of contributors. Through interviews, artistic leaders engage in conversation about audience, community, and the value of art. Beyond these thoughtful essays, the book includes the results of a two-year study titled, Understanding the Intrinsic Impact of Live Theatre. This study, composed of patterns of audience feedback in 18 theaters and 58 productions, was commissioned by Clayton Lord of Theatre Bay Area and was completed by Alan Brown and Rebecca Ratzkin of WolfBrown.
In the first chapter, Clayton Lord, sets the stage with the state of valuing and evaluating the impact of art. He suggests that nonprofits are asked to “justify their existence by talking about two things: anecdote and economics” (p. 24). Instead of continuing on this path, Lord calls for a return to valuing and attempting to measure the “unmeasurable” parts of art, despite the difficulties in doing so. The two-year study of the intrinsic impact of theater, based on WolfBrown’s metrics described below, offers a starting point for standard metrics and a common vocabulary to “talk about the intangible power of theatre and art” (p. 37).
Chapter 2 details the Brown and Ratzkin study on live theater. The researchers use three constructs of “readiness to receive” and five constructs of intrinsic impact to interpret the survey data. The five constructs used as metrics for intrinsic impact included intellectual stimulation, emotional resonance, aesthetic enrichment, social bridging, and bonding, were based on WolfBrown’s 2007 work, Assessing the Intrinsic Value of Live Performance. The results of the live theater study are not discussed thoroughly by the researchers. Instead, the reader is led to draw his own conclusions with a return to conversation and essays with artistic leaders.
In the third chapter, twenty artistic leaders engage in interviews to address how audience feedback influences their work. Additionally, the conversation spans to how theater can influence civic discourse. Michael Rohd of Sojourn Theatre rarely administers surveys to understand audience feedback and impact. Instead, he is interested in knowing how the spirit of the project has continued after the performance, as may be demonstrated in the actions of community members and organizations.
In sum, this piece offers an approach to collect and interpret the intrinsic impact of art, specifically within the context of live theatre. The contributing artistic leaders offer examples of successes and challenges in this process, as well as thoughts for moving forward.
See further reviews of this work by Grantmakers in the Arts and the National Endowment for the Arts.
Marty Pottenger discusses Art at Work in Portland, ME and the evaluative practices that were used in assessing participant impact. This resource captures Pottenger’s contribution from Animating Democracy’s 2012 Social Impact & Evaluation Blog Salon.
Dance practitioners across the country are creating innovative opportunities for community, civic, and social engagement. Choreographer, organizer, and former Urban Bush Women company member Paloma McGregor highlights contemporary community-based dance practice; concert dance that is intentional in connecting to community members and issues; and programs where the next generation of socially engaged dance artists are incubated. Through a wealth of stories and examples of dance artists all across the country, McGregor describes how community-based dance—the work of both pioneering companies and fresh new artists working in collaboration with community partners—bridges boundaries between individuals and communities, and between communities. Concert dance is pushing new boundaries by not only taking on social justice issues but fostering community dialogue, bringing awareness to issues of global significance, and demanding visibility for communities who have historically been overlooked. And incubator spaces are fostering community building while helping shape the future of dance practice and reveal its relevance and relationship to communities. In a variety of community settings––art centers, jails, studios, schools, commercial buildings and more––these new practitioners find ways to help people connect with their own physical experiences, to engage in the civic and social life of their communities, and to discover a new vision of their world.
Grounded in a recent strategic plan, the Tucson Pima Arts Council is moving to advance civic engagement in the city and county through its programming, funding, and partnerships. As part of Animating Democracy’s Art & Civic Engagement Impact Initiative, and in addition to the qualitative focus reflected in the evaluation inquiry with Maribel Alvarez, TPAC wanted to know what concrete measures are reasonable to use to understand the civic engagement effects of its work as an agency. The objective of the collaborative inquiry with Mark Stern and Susan Seifert of the Social Impact of the Arts Project was to develop a plan for systematically collecting and analyzing data on civic engagement.
The report begins with a review of the policy context in which the collaboration was undertaken, including the findings of the TPAC strategic plan. This is followed by a discussion of the proposed strategy and specific recommendations for documenting civic engagement. Specifically, Stern and Seifert propose five strategies: improving organizational data gathering, telling stories, documenting artists and the informal cultural sector, identifying institutional networks, and using geographic information systems to integrate data for analysis. These recommendations provide a plan for documenting civic engagement and the arts. However, given the fiscal and social realities, the third section of the report outlines an implementation plan that would allow for staging these elements depending on the resources available.
Native American artists and culture bearers brought Indigenous perspectives and critical voices to pressing issues of the environment at the April 2012 conference, Echoes of the Earth in Times of Climate Change, sponsored by the Seventh Generation Fund and Hopa Mountain. Artist, writer, and activist Edward Wemytewa (Zuni) eloquently captures the perspectives of Native leaders and culture bearers as they look to their cultural heritage and wisdom—sacred ceremony, ancient languages, prophesy, and hallmarks of mutuality, reciprocity, and responsibility—for ways to regain the delicate ecological balance of the earth.
