The Policy Machine
The dangers of letting algorithms make decisions in law enforcement, welfare, and child protection.

In December 2007, Indiana resident Sheila Perdue received a notice in the mail that she must participate in a telephone interview in order to be recertified to receive public assistance. In the past, Perdue, who is deaf and suffers from emphysema, chronic obstructive pulmonary disease, and bipolar disorder, would have visited her local caseworker to explain why this was impossible. But the state’s welfare eligibility system had recently been “modernized,” leaving a website and an 800 number as the primary ways to communicate with the Family and Social Services Administration.

Perdue requested and was denied an in-person interview. She gathered her paperwork, traveled to a nearby help center, and requested assistance. Employees at the center referred her to the online system. Uncomfortable with the technology, she asked for help with the online forms and was refused. She filled out the application to the best of her ability. Several weeks later, she learned she was denied recertification. The reason? “Failure to cooperate” in establishing eligibility.

An algorithm is a set of instructions designed to produce an output: a recipe for decision-making, for finding solutions. In computerized form, algorithms are increasingly important to our political lives. According to legal scholar Danielle Keats Citron, automated decision-making systems like predictive policing or remote welfare eligibility no longer simply help humans in government agencies apply procedural rules; instead, they have become primary decision-makers in public policy.

Read more...