Weapons of Math Destruction

While technology can seem like a way of eliminating prejudice from a variety of domains, it often encodes human biases and prejudice. One way it does this is by applying a specific set of rules to a large group of diverse people, effectively stripping away consideration of their individual needs and circumstances. By forcing many people to conform to a single model, outliers and complex situations are lost in the shuffle and suffer the consequences. It may seem fair that the same model or equation is applied to everyone, but the lack of consideration for the unique situations people face makes that equation unbalanced.

Beyond scale, machines are also biased in that they can only see what their creators have programmed them to see. In the case of Weapons of Math Destruction, or the algorithms that determine how much of our society functions, they are often created by a select group of wealthy individuals who work in the financial or Big Data sectors with privileged backgrounds and top tier educations. While their intentions aren’t necessarily to create bias, they do seek to yield profits. When people in this elite class are the ones making the rules and applying them to everyone, the machines can’t help but inherit the bias their creators harbor, knowingly or not.