there are, again, processes for holding humans accountable.
we work very hard to limit people’s access to these, because they’re expensive to operate, and we give a very large leeway to humans because it’s incredibly hard to prove one way or another whether they were discriminating.
I think it’s funny that everyone has jumped to hiring algorithms, to the universal horror of dealing with an automated hiring system—in silicon valley we don’t have those, we barely even have recruiters, it’s common for a nontrivial percentage of recruiting to go through personal contact with the CEO even as companies get enormous, and the rest to go through direct email to hiring managers.
Perhaps the real fault line is that Gus and I believe that it’s possible to create an algorithm that does what it says and to intelligently evaluate it before putting it into practice, and your assumptions are that this is borderline unheard of, that any algorithm you end up in contact with will be snake oil that merely adds to the bureaucratization of everyday life. It is true that I’ve heard more stories about terrible hiring systems in europe.
This is, of course, why I argue vehemently that there is a need to make algorithms auditable by the public, and surely some safeguards or legal standards are suitable before putting them in charge of important life decisions. More directly, if you put a racist algorithm in charge of a hiring process, surely you should face the criminal penalties just as if you did it yourself.
And finally, and this may come as a shock to those of you imagining the americans as the handmaidens of capital, I know that I have no desire to share any of my data with anyone. Nobody has any right to track me. More or less, in any situation. I think we should shut down the advertising industry entirely on the grounds that it interferes with people’s self-determination, not just the targeted bits. Abolish billboards, abolish magazine ads, abolish facebook ads, delete it all.
This all said, if you think there’s something wrong with a use of algorithms, that exists well before you encode them onto a computer. It goes to the root of the legal system—itself an algorithm.