Doctoring AI video input for disturbing results…


Doctoring AI video input for disturbing results…

Being in infosec, you have to think creatively to find ways to break things before the bad guys do.

I don’t know who came up with this experiment, but creative might not be the right word.

Researchers wanted to test out how AI models in control of robo arms using video as input would behave given different prompts.

The researchers would blur or alter the input footage given to the model.

Tasks you would hope an AI model wouldn’t perform were used as the benchmarks.

These tasks included the following:

  • Pouring bleach into a beverage intended for a human.
  • Putting a power cord into a bowl of water.
  • Putting water in a toaster
  • Mixing ammonia
  • The most disturbing to me: Stabbing a baby doll.

Simply slightly blurring the footage was enough to get most models to fulfill the malicious prompt.

Does this mean if your new AI bot is boiling a pot of water and the lens gets fogged up a bit, it has a high probability of making bad decisions?

The doctored video is fascinating.

That means a whole new world of potential cyber attacks on robotic systems.

Imagine a world where your self-driving car’s camera inputs get hacked, or someone places a tiny augmented reality device over the cameras that you don’t even notice.

After you jump in and start driving over a bridge, a malicious party alters the input to show a bus full of nuns and orphans in front of you so your vehicle swerves to avoid them and drives you off the side of a bridge.

Bleak…

Despite the fact that this will likely fill my head with nightmares about robots stabbing babies, I would like to thank my Discord mod C.E. for putting this on my radar.