AI is a Real Threat to Human Civilization. Let Me Explain Why

I started thinking about all the technology that already exists in manufacturing, and how combined with such powerful AI, this threat against humanity is really non-zero. And I am now convinced that it is truly a bigger threat to human civilization than we all think.

The Danger of Optimization Without a Moral Compass

For years, manufacturing has worked on optimization of production processes using technological developments to do so more efficiently. We are always working toward improving quality, reducing costs, and ensuring our assets are performing at optimal levels. Humans were optimizing to improve profitability, but with guardrails around what was moral.

At first glance, it seems that AI and machine learning can further these goals. Yet, it’s important to remember that AI is simply trained to optimize something; it has no moral compass to determine if that optimization is good or bad. AI’s primary goal is to achieve a programmed outcome. We can program it to have a moral compass, but it depends on whose moral compass we use.

The danger comes in if the programmed optimization is either misinterpreted by generative AI technologies or if someone intending harm initiates an inappropriate or destructive optimization. For example, I think it’s likely that a manufacturing firm could accidentally optimize the production of dangerous or defective products that could harm humans or the environment or simply rapidly deplete our natural resources in the name of efficient manufacturing.

Manufacturing Control Systems Are Now at Greater Risk

Plenty of bad actors out there are already initiating sophisticated cyber attacks on companies, organizations, and people around the world. I think as more manufacturing processes become automated and take advantage of cloud technologies, the risk of cyber attacks increases.

Right now, many manufacturing facilities rely on programmable logic controllers (PLC) to automate specific processes. I think it’s entirely possible for someone armed with AI tools to break into these control systems and cause a lot of damage. For example, I think AI might be used to shut down production of a competitive business or even cause intentional build-up of dangerous gasses or pressures that could result in life-ending explosions in industries such as oil and gas.

Unsupervised Lights-Out Manufacturing Could Lead to Autonomous Destruction

Some companies are already utilizing lights-out manufacturing, or a fully automated production facility that creates products with little or no human intervention. A Japanese robotics company has been operating a lights-out facility since 2001, utilizing robots to build more robots, and can run unsupervised for 30 days at a time.

What happens if one of these facilities combines its lights-out manufacturing technology with generative AI that determines an “improvement” in the process should revolve around the creation of dangerous or harmful products? The development of autonomous weapons, nuclear arms, or biotechnology threats on a large scale are entirely possible without appropriate safeguards.

Ushering in a Whole New Era

In just the last few months, we’ve seen the massive effect that generative AI technologies may have in our industries, society, and world. When these threats and dangers cross over from the purely digital world into our physical one, we can see that AI has the power to not only revolutionize life as we know it in a positive way, but also carries the potential for grave danger and even the possibility of the extinction of human life.

While I don’t believe it’s possible to stop AI’s development, it will be exceedingly important to be cautious moving forward, both in terms of future research and application as well as creating safety and security measures. It’s critical that humans remain in control and that this technology is used for the benefit of humans, not to lead to their destruction.