August 02, 2006
Beating code with code
CAPTCHAs are a great example of a clever incentive-centered design for an information world problem. But, as many people point out, they aren't perfect. Matt May at W3C has a nice slide presentation explaining CAPTCHAs and a number of their accessibility problems (based on a nice paper for those with more taste for details). He also discusses a variety of ideas about how to do better. Clever as some are, they all suffer a common problem: the incremental improvement from each is largely a technological fix, not an improvement in the incentive structure of CAPTCHAs. And technological fixes in this area are doomed to fail approximately equally rapidly.
What do I mean by this? The costs of computing cycles are falling exponentially, and the implementing usable clever algorithms is probably falling at a slower but still exponential rate (if for no other reason than a big part of the cost is the enormous computational power needed for some tough problems like password cracking and automated visual recognition of CAPTCHAs, etc.).
Technological fixes are just a loop in an arms race. CAPTCHAs, for example, grew out of the observation that automated visual recognition of distorted alphanumerics was pretty poor a few years ago. But now, largely in response to CAPTCHAs, automated breaking has rapidly advanced, and CAPTCHA security is getting rather weak (which is why it's used only to protect relatively low value resources).
Unless we identify a human cost (or more precisely, difference in cost between good guys and bad guys, a difference we can use to distinguish between them), and design incentives around that cost (or benefit, if you want to flip the sign bit) tech fixes will be very short term and their efficacy will decrease rapidly. Incentive-based solutions can be more durable if they are based on features of humans or their utility functions that are are not subject to technological end-runs. It's true, it's not always easy to find incentives that aren't susceptible to end-runs, but it's not hopeless. Money works pretty well in many cases; sure, technology (i.e., counterfeiting) can sometimes do an end-run, but the rate at which technology has been making money obsolete as an effective incentive is a whole lot slower than pattern-recognition software is advancing on CAPTCHAs and the post-CAPTCHA fixes that W3C discusses.
Posted by jmm at August 2, 2006 12:45 AM