AI algorithms provide the news we read, the ads we see, and in some cases even drive our cars. But there’s something very wrong with these algorithms: They rely on data collected by and about people, and they can confirm our worst fears about biases. For example, job-screening algorithms can automatically reject names that sound like they belong to non-white people, while facial recognition software is often much worse at identifying women or non-white faces than it is at identifying white men. A growing number of scientists and institutions are reporting these issues and talking about the potential for AI to cause problems.
Brian Nord is one such researcher who speaks to the potential for harm from AI algorithms through his own example. Nord is a cosmologist at Fermilab and the University of Chicago, uses artificial intelligence (AI) to study the universe, and is exploring an idea for a “self-guided telescope” that can write and make predictions with the help of a machine learning algorithm. At the same time, he struggles with the idea that his algorithms might one day be biased against him—and even used against him—and is working to create a coalition of physicists and computer scientists to gain more oversight over the development of AI algorithms.
Brian Nord says that creating the coalition will be a way to clarify potential harm beyond race and ethnicity and show people that they fully understand the problems or biases that can be created.
Recently, an algorithm was created that used neural networks to try and speed up the selection of candidates for doctoral programs. They trained the algorithm with historical data that included data about applicants who were rejected and accepted to universities in the past. These candidates were selected by professors and people with biases. It is obvious that anyone who develops this algorithm will encounter these biases and will have to correct.
Brian Nord continues by saying: “We can’t predict all future uses of technology, but we need to ask questions at the beginning of the processes, not later. A responsible person could ask these questions, allowing science to complete its work, but without putting people’s lives at risk. We also need policies at various levels that make a clear decision about how safe algorithms need to be before they are used on humans or other living creatures. We have seen cases where a homogeneous team develops an application or technology and didn’t see things that another team that wasn’t there would have seen. We need people with diverse experiences to participate in designing policies for the ethical use of AI.”
Brian Nord’s biggest fear is that people who already have access to technological resources will continue to use them to subjugate people who are already oppressed. Pratyusha Kalluri has also mentioned this idea of power dynamics, saying that we see this all over the world. Sure, there are cities that are trying to ban the use of facial recognition, but unless we have a broader coalition, we won’t be able to stop this tool from making things worse – white supremacy, racism , and misogyny are just some of the things this tool will exacerbate. Pratyusha Kalluri goes on to say, “If we don’t advance policies that prioritize the lives of marginalized people, then they will continue to be oppressed.”
Information source: gizmodo.com
