|
|
"Outcome measurement" is a popular topic in health and human services organizations these days. But what does it really mean? Of course, "outcome measurement" means measuring and quantifying the impact of one's services on those who receive them. Equally important and much less discussed, there is an art (as well as a science) to outcome measurement. And the difference between doing it artfully and doing it by rote can be the difference between measures that are meaningful and those that are merely an expensive waste of time.
I recently read an article in the 03.25.00 Philadelphia Daily News entitled, "For 2 Milwaukee families, it's hard times revisited", written by Joanne Weintraub of the Milwaukee Journal Sentinel. The article was about a documentary produced by Bill Moyers, chronicling the experience of two families beginning in 1991. The purpose was to track the impact of the nation's booming economy on these small economic units. Unfortunately, that impact could best be described as "none".
The phrase that caught my eye was, "The fact that prosperity has left many families behind 'is reported, but it's mostly reported as statistics,' Moyers said recently in a phone interview. 'What we've wanted to do, and what I think we've done, is to translate it into human terms.'"
Not very long ago, my job was to sell software to Health & Human Service agencies grappling with the question of how computers could help them measure outcomes. As we all know, donors are requiring better reporting on how their dollars are being spent. Outcome measures are the group of statistics these agencies use to evaluate a program's impact. One of the things I heard repeatedly from potential customers was "Do outcome measures accurately convey the impact our programs are having on clients´ lives?" Though these doubts were widespread, they were rarely discussed because the reporting requirements were a fait accompli. "Opting out" was not an option. In any case, the organizations were simply too busy designing, purchasing and implementing new systems to worry about whether they were 'painting' a realistic portrait of their clients´ experience.
While working with the elderly some years ago, it was my job to write progress notes for clients who attended the outpatient day program. Each worker had a caseload of 50 - 60 clients, of whom about 20 attended on any given day. Our notes were combined with notes from the social worker, psychiatrist, and nurse, and were used to assess client progress. To say that we were often overloaded with day-to-day tasks, and that our progress notes were scant, is an understatement.
Could we have benefited from sophisticated computer equipment to better document our daily work? I think fast computers and easy-to-use software would have speeded the documentation process, but they wouldn't have addressed the "note quality" problem. To thoroughly measure the impact of our personal involvement in the lives of our clients, on the other hand, could have become an end in itself. Moreover, statistics may show changes in the overall performance of a group of people, but they can't tell you much about the impact of your work on one life. Not about the smiles or tears of an individual for whom you have made a difference.
Realistically, how much can you measure well? I would argue that just like with computing--or even brain--power, we should be judicious in choosing what to measure. We can't measure everything, unless we intend to spend the bulk of our time measuring rather than performing service.
I am not a Luddite, I'm not even a Clifford Stoll, but I think those who deliver service should have a say in shaping measurement programs. What is reasonable? What is realistic? What can be done to capture some of the more qualitative elements of the service relationship? While it is entirely fair for donors to expect some reporting, perhaps non-profit staff can help them build some of the "art" back into the process without constructing a cumbersome, earthbound blimp of a measurement program.
As I have said before and will say again, people who work in non-profit organizations are caring, intelligent and resourceful. When their ideas are applied to "the measurement problem," innovation is sure to result.
by Michael Feagans
|
|
