A good illustration of that problem is a Microsoft experimental AI chatbot that became both racist and anti-feminist after it was fed with Twitter conversations for just one day.
Another example is the AI tested by the American police that directed officers to disproportionately patrol African-American neighborhoods after consuming “dirty data” from decades of unjust police actions.
Another challenging aspect of AI is machine ethics. Because of the unprecedented pace of new technologies, machines making life-and-death decisions are only a matter of time. Just imagine a self-driving car deciding who gets to live: the kid who suddenly runs on a red light in front of it or two senior citizens in the vehicle who will die in case of maneuver.
But, probably, the greatest concern is Singularity – a hypothetical point of technology where AI might become uncontrollable and irreversible in its actions, resulting in unforeseeable changes to civilization as we know it.
At the current state of AI, we are far from a real danger, but the question remains. How can we know that strong AI will be good to us?