Pinpointing
the Effects of Community-Centered Practice
An address delivered at the
Community Centered Practice Learning Summit
New Orleans, LA
October 15, 1998
Thomas E. Lengyel
Director of Research and
Evaluation Services
Family Service America
Define what you mean by community building,
community organizing, economic development, or whatever you call the thing you
do.
Define the goal(s) of your actions as specifically
as possible, and independently of how the goals may ultimately be
measured. Goals do not have to be
directly observable.
This is the hardest
part. Spend time on it. Don’t put the issue down until you can
explain it easily to a lay person, like someone from United Way.
Example: “What is social participation?” Lesson:
Indicators can be identified, even for something which seems nearly
undefinable. (See below.)
Describe, at least for yourself, the chain of cause
(your “theory”) which leads from goals (e.g., increased social capital) to
observables (e.g., relative presence of graffiti).
C. Define the setting where the desired effects will be the
strongest, and measure there
Participants with greatest exposure before
those with less exposure (dose)
Participants with greatest opportunity for gain
before those with less opportunity before those with no opportunity
Any participant before any non-participant
Block, group of residences, or specific households,
before neighborhood (the specific before the more general)
Neighborhood, before community
Community – BE CAREFUL! (Don’t even go there.)
Your own organization.
Don’t forget this. Commitment
to the development of social capital should also have organizational and
personal effects on facilitators.
Leave sampling weaker settings for later, or treat
them as confirming
Does the rate go up or down? Significantly?
A rise in reports of crime
in a neighborhood might be evidence of increased social capital among the
residents, or it might mean the neighborhood is deteriorating. Ambiguity is often resolvable with more
information (e.g., who did the reporting?).
Many, many individuals, organizations, and branches
of government busy themselves collecting volumes of information about
consumers. Much of this is high quality
and can be used to document changes in individuals, families, and neighborhoods. Some of these sources are schools,
hospitals, the Health Care Finance Administration, state and local health
departments, police, libraries, local political parties, community action
agencies, county department of social services, YMCA, United Way, etc.
Particularly useful are data that are kept in the
normal course of business, and data that are periodically renewed (e.g.,
through regular re-surveying).
Use social indicators freely. E.g., voter participation (others make it
their business to keep such records, so you don’t have to gather new data).
G. Kill only what you can eat.
Collect only information that will definitely bear on a necessary
decision. Never ask anyone,
particularly a participant, to report or collect information because you
think it might, one day, be useful.
One or two outcomes per program are plenty.
Interns are energetic and often low cost. Schools and public libraries have also been known to take on
basic research projects, including inventorying neighborhood resources.
I. Use multiple approaches, if time and money allow
(1) Non-standardized measures: Attainment of project goals
Be bold, creative.
Make you own instrument or survey.
Who told you you couldn’t?
(2) Standardized instruments (normed, psychometric
instruments).
(3) Surveys of application of knowledge, tools, or
skills (not “customer satisfaction”)
Ask: How would
I know if someone used or applied something that I showed, taught,
facilitated, or encouraged?
Then: Compose a survey, which
asks participants if they did so.
II. Evaluating change in
your organization
To use business records to gauge organizational
change, you must first have a clear understanding of what you’re looking
for. In this case, you must develop a
clear concept of community-centered practice.
The business records of the participant offer
testimony as to their prior involvement with community-centered practice. Dated historical records which contain
useful information, and associated questions, include:
(1) Previous mission and goal statements
Do they refer to building community resources,
collective capacity, or developing community norms?
Has there been any recent change in mission and goal
statements related to community-centered practice?
(2) Annual reports
Which organizational activities are promoted?
To whom does the organization customarily give its
Annual Report?
What degree of prominence is accorded to community
building? E.g., are revenues and
expenses for community building separately broken out?
(3) Marketing campaigns, press releases, public
relations materials
What does the organization sell?
To what degree does the member organization market
itself in terms of community-centered practice?
(4) Fund raising
Where does the organization get its money?
Who pays for community-centered activities and
programming?
What portion of the budget is for community-centered
activities?
(5) Job descriptions
To what degree do they refer to knowing community
resources, invoking or augmenting social resources, strengthening community
norms and expectations, establishing new linkages, or building community
capacity generally?
(6) Employee performance reviews
Are staff members evaluated in terms of their
knowledge of and contribution to individual and community assets, capacities
and norms?
(7) Staff training
Has the organization sought to train its staff in
skills necessary to community building (e.g., community organizing, community
surveying, economic development, entrepreneurship, home ownership programs,
community collaboration, etc.)?
(8) Reward, merit, and advancement policies and
practices
Are rewards given for demonstrated skill and
results in community organizing, asset mapping, strengths-based assessment, etc?
III. Evaluating Street-level
effects
(2) Individual asset mapping (FSA Assets and
Capacities Inventories)
Assets Inventory can also be used to discover
community assets (e.g., by asking participants to name the assets they use)
Risk is that you will impose your own view of
what are potential resources
Risk is particularly acute in situations where there
are significant social differences between the evaluator and the residents
(1) Powerful technique, relatively inexpensive.
(2) Plan what you’re going to do with the
information before you structure the questions. Ask questions only because the answers provide needed
information. If the answer to a question doesn’t fill a specific need, don’t
ask it.
(3) Anticipate ambiguity and misunderstanding in
responses. Field testing helps with
this.
(4) Trade-off between ease and richness of
information. The more constrained the
choices of the respondent, the easier to administer and score. But ... the more canned and unspontaneous
the data. Less chance of discovering
something new or unexpected, more chance of predetermining the pattern
through your own prejudices.
(5) Determine the history of surveying with this
neighborhood or group of participants and be sensitive to it. Have they been surveyed before? Recently?
Were promises made about what the surveying would lead to?
(6) Strive for good, not perfect
representation. Forget about
randomization of samples. Often you can
just try and survey everyone (our contexts are usually small enough). Then, review what bias may exist in your
admittedly not fully representative sample.
(E.g., Who was most likely to cooperate with me? Those who were home and not working? Those who were most favorably inclined to
the project?)
(7) If you survey subjects via self-reports, ask
them to report their actual behavior, not their evaluation of their behavior.
Ask: How many
times did you leave the house alone at night during the past 7 days?
Not: Have you
been feeling more or less safe at home after the incident/training
program/counseling session etc.?
Inexpensive.
Very good at teasing out “culture”, i.e., values, norms, and
expectations.
Requires a trained and experienced facilitator,
preferably from the ‘hood.
A major risk here is bias in who gets included. It’s often very hard to know who the voices
in a focus group represent. Sometimes
they made be simply the most visible or outspoken individuals.
This involves participating in the life of some
community on a putatively equal footing with the natives. (This is a polite fiction: It is never really equal.) Try to have the same experiences, on the
same terms as the residents. Describe
what is going on from this “insider” perspective.
This is a good way to understand the motives of
actors and how circumstances impinge on them.
It is also surprisingly hard to write up, for the simple reason that
description is so open ended. So, if
you want to know why people aren’t on the streets of the neighborhood after 8
p.m., hang around and you’ll find out.
There are more available than you think. Low birth weight, school attendance, many types of crime and
arrest reports, voting rates, home ownership rates, tax rolls, birth
certificate databases, hospitalization records (actually available in certain
forms), census records of income and educational levels. Many of these are address specific, so you
can measure more or less where you choose, or at least down to the block level.
Definition: Community building
means the stimulation and enhancement of social capital. Social capital is increased when social
organization (connectedness) is strengthened, consensus about values grows,
common expectations or norms about conduct emerge, and when members of a
community accept and seek ownership of their own community life.
Hypothesis #1:
Increased social capital is manifested in increased social participation
(1) Voting rates:
Increase, decrease or
flat? Must be contextualized (i.e., viewed
in context of general changes in voting pattern in surrounding community).
(2) Competition for local
offices
Do people compete for lower
political offices?
(3) Church participation
Increase, decrease, or
steady?
Relate to posture of
church. Is the church engaged with the
community?
(4) Associational life
Membership in associations
and clubs
(5) Neighborhood
participation in and organization of cultural rituals
Trick-or-treating, caroling, the Gwinnet “walking
group”, housewarmings, open houses, birthday parties, Kwanza
Hypothesis #2:
Increased social capital is manifested in resident involvement in public
order and security
(1) Robustness of street
life for residents, particularly morning and evening (pre/post video?). Is the street a “place to be” vs. be
avoided?
(2) Responses to trash and
graffiti (pre/post video?)
(3) Complaints to police,
particularly around drugs, prostitution, noise, aggressive panhandling, access
to & use of public spaces (who complains?)
(4) Community policing
Is it requested, or
supported?
(5) Occurrence of community rituals around order
“Take Back the Night”, the Gwinnet Walking Group
(6) Social organizational forms focusing on order
Neighborhood watches, Block Parent participation
Hypothesis #3:
Increased social capital is manifested when neighborhood values attract
discussion and debate and move toward consensus
(1) Evidence of shared values emerges in
meetings and community forums (including informal forums like the barbershop
and tavern)
(2)Events express consensus
about values
People’s Park, community
gardens
(3) Shared feelings among
neighbors (need to define)
Development of neighborhood institutions??
Hypothesis #4: Increased social
capital is manifested when residents, neighborhood associations, organizations,
businesses and institutions are linked by stronger reciprocal bonds, including
resources outside the neighborhood
(1) Assets inventories
demonstrate links with resources “downtown”, external to the neighborhood.
Social support vs. leveraged
social capital; “getting by” vs. “getting ahead”
(2) Repeated Asset
Inventories show an increase in the variety of used assets among project
participants.
(3) Repeated Asset
Inventories show an increase in the helpfulness of used assets among project
participants.