Bias Built into an Algorithm

An algorithm counts as biased when its outputs are systematically skewed against a protected group, like race, gender, or age. That skew does not require the algorithm to ever check that attribute directly. A proxy variable is any input that correlates with a protected characteristic even though it is not that characteristic itself. Zip code correlates with race because of historical housing segregation. Name correlates with gender and ethnicity. College attended correlates with family income. When an algorithm weighs these proxies, it reconstructs the protected characteristic indirectly and bakes its bias into a decision that looks purely numerical.

Training data makes this worse, not better, in systems that learn from the past. A hiring filter trained on a company's past hiring decisions treats those decisions as the correct target to reproduce. If the company's human recruiters favored certain schools, names, or neighborhoods for decades, the algorithm learns that pattern as the definition of a good candidate and repeats it at scale, often more consistently and less visibly than a human would.

This is why arithmetic is not the same as neutrality. An algorithm is a procedure built by people, trained on data generated by people, operating in a society already shaped by unequal patterns. Numbers do not launder that history away. This connects to the next idea in the unit: once bias is built in, auditing the algorithm's outputs across groups, not just inspecting its code, is often the only way to catch it.