09.09.08
Metrics Matter
The Pitfalls of Linking Doctors’ Pay to Performance
It’s a difficult issue. While the metrics and pay-for-performance programs profiled in the article are obviously crappy, and by all accounts antibiotics are grossly over-prescribed, it’s important not to take that as an indictment of metrics and data altogether.
A lot of professions fall into this trap where we say, “well, [medicine/law/teaching/programming] is really complex, and it’s really hard to design good metrics, so why don’t we just bank on the fact that the professionals in this field are really good people who work really hard and have the best of intentions, and it’ll all work out for the best.” That’s all true, but it also ignores the fact that data - particularly in the aggregate - can help us make better decisions. And sometimes - not always, but sometimes - this data goes against conventional wisdom, or individual expertise.
One of the neat things I learned from one of the Gladwell books or articles (can’t recall which one) was how important the Apgar score proved in improving neo-natal care. Simply by having a consistent and statistically useful measure of how healthy a baby is (even if it papered over or ignored the individual complexities of each situation), doctors and researchers were able to learn from thousands of cases and derive best practices. Were those best practices always applicable? No. But they were a whole hell of a lot better than what we had before.
By and large, people have an anecdotal bias - we tend to learn and internalize the lessons of our most memorable experiences, with far too little heed given to unbiased data that all too often better reflects reality. Should we draw a straight line from data to compensation? No. Systems like that are easy to game and not particularly helpful. But at the same time, we shouldn’t minimize the importance of gathering good data, developing valid metrics and - without putting all of our eggs in one basket - using it as a component in compensation.
[via Marli]