David Henderson
Los Angeles, CA
RSS




Wednesday, January, 27, 2010
Great Non-Profits Deserve A Great Rating System




We have an evaluation problem in the social service sector.  We want evaluations to be easy more than we want them to be right. Designing good surveys and collecting client data is hard. Rating how we feel about a particular program on a scale from one to five is easy.  As a sector, we need to guide funding towards programs that work, and abandon ones that don't. If we are to reliably move resources towards the highest achieving organizations, we have to define what high achieving means.

To me, the answer to what makes an organization high achieving is clear.  The social service sector exists to reduce social ills like poverty, homelessness, and food insecurity.  Organizations that have a greater impact on improving the lives of their clients are better than those that have less.  Any evaluative framework that is not centered on measuring changes in client indicators is irrelevant. Despite this obvious point, I am dismayed by how celebrated efforts like the Alliance for Social Investing and Greatnonprofits.org fail to base their evaluative criterion on client outcomes. 

There is a lot at stake in getting a rating system right (or wrong).  The potential harm a poor rating system can cause was illustrated last week in a partnership between Greatnonprofits.org and Guidestar.org.  These two rating organizations teamed up to compile a list of the "Top Ten Relief Organizations Working In Haiti."  The list was compiled based on a handful of donor reviews, and as non-profit consultant Gayle Gifford pointed out, those organizations "that were listed in the Top 10, had ONLY 1 or 2 Reviews. That's it."

Greatnonprofits.org and Guidestar.org responded to Ms. Gifford's criticism by dropping the top ten list all together.  While these rating organizations certainly did the right thing by retracting their list, it is amazing to me that two supposed evaluation leaders in our industry could have compiled such a hasty, pointless agency ranking in the first place. There is so much that is problematic here, least of all the paltry number of reviews the top ten list was based on.

If we are to ever develop a meaningful top ten list of the most effective social service programs, we have to embrace the social scientific complexities of evaluating clients' social outcomes.  This means taking the collection and analysis of client data, in its quantitative and qualitative forms, seriously. Simplistic rating systems that ask donors how they feel about a particular organization may seem seductive, but they could not be more beside the point in determining which organizations are best able to improve the lives of hurting people.  So long as we fail to move towards an evaluative framework that is centered on sound social outcomes practices, the only top ten list we can reliably compile is the "Top Ten Worst Ways to Rank Non-Profits."

(Photo by surfspirit)



Steve Albertson: Good post! Like any user rating system, these sites rely on a vast "market" of reviewers to provide a representative assessment of what a large number of people think about something. Large numbers help to minimize the rating impact of outliers (people who blindly love or blindly hate the organization), and ideally you get enough people who have actual experience with the organization to give an informed opinion. Unfortunately, without large numbers of reviewers, simplistic rating systems just aren't very representative. One can dig further to read detailed reviews and perhaps form opinions based on what is written, but the raw "star" rankings probably won't be that accurate.
Posted 2010-01-27 11:23:18
UnionBank - Supports Ending Homelessness
Linda Rosenberg, MSW
Saving Jobs, Saving Public Dollars: Intervening Before Disability
Linda Rosenberg, MSW
Juan was a delivery driver, but his health problems were putting him at risk of losing his job. His diabetes was poorly controlled and had caused foot ulcers that made it difficult for him to walk. He also had bipolar disorder, which was not being controlled. When he joined the Working Well program in Harris County, Texas, Juan worked with a case manager to get orthopedic shoes, to receive support in developing a diabetic diet and exercise plan, and to make an appointment with a psychiatrist to bring his mental health condition under control. As a result, Juan was able to continue working full time as a delivery driver and received a raise for exceptional performance (Bohman, Stoner, & Chimera, 2009). Working Well is part of the....
Homelessness Resource Center
Digging for Treasure Together: The Spirit of Motivational Interviewing
Homelessness Resource Center
"I believe that people who are trained in Motivational Interviewing have a different perspective on building relationships. Many people are trained to view clients in a paternalistic manner, like they are children. This attitude says that providers always know what is best for a client. This attitude makes us think that we must convince clients that we know what is best for them. In this view, success is measured by the provider’s actions, not the client’s." In the spirit of Motivational Interviewing, Alan Pickett, a Mental Health Outreach Nurse at the Project Outreach Team (PORT) for Washtenaw County Community Mental Health in Ann Arbor, Michigan, does not define success this way. He believes his first task to is to make himse
David J. Jefferson
Take Action and Confront Our Fear of Poverty
David J. Jefferson
“The prevalent fear of poverty among the educated classes is the worst moral disease from which our civilization suffers.” When William James, ostensible father of American Psychology, penned this line over a century ago, he had embedded the idea in a discussion on religious experience, in which he also extolled the virtues of voluntary poverty. Today, in a totally distinct context, James’ words are imbued with new meaning. Due to economic recession and what many economists have deemed a jobless recovery in contemporary America, people who never before feared poverty—the educated, or middle class—must now confront harsh new realities. The current conditions are like nothing we’ve ever seen. Conspiring myriad circumstances....
Jamie Van Leeuwen
Denver Defies Poor Economy to Help Homeless People
Jamie Van Leeuwen
When Denver’s Road Home began over four years ago, we never could have anticipated a year like we just had. Who could ever imagine that our economy would shift into a global recession unlike anything the country has experienced since the Great Depression? And yet, in the midst of unparalleled economic shifts, there comes great opportunity. We believe there has never been a more important time for Denver to have a plan to end homelessness. During the past year, our homeless plan has been tested and we expect the coming year will continue to pose new challenges for us. Housing foreclosures, unemployment and funding cutbacks will continue to place new demands on our ten-year plan to end homelessness. As a result, we updated our plan so....
Linda Valverde & General Dogon
When Revitalization Becomes Gentrification
Linda Valverde & General Dogon
The authors have lived and worked in Central City East—commonly known as Skid Row—for a combined thirty years. During all of our decades of living in Los Angeles, neither of us had ever heard of gentrification until about five years ago, when we became members of the Los Angeles Community Action Network (LA CAN). Now it seems that gentrification has become a common household word in cities throughout the United States, and nowhere more than downtown Los Angeles. When redevelopment really took off in downtown L.A. in 2002, LA CAN and our allies created five principles for fair redevelopment. We wanted to see our neighborhood revitalized, not gentrified. The principles were....






Inforum RSS Feed