1 hour ago
In July, Hugging Face, a institution that hosts overmuch of nan world’s AI package and open-source AI models, was hacked. A malicious dataset had been utilized to tally codification connected 1 of its servers. Whoever was down it captured soul information credentials and moved done systems complete a weekend, moving thousands of actions from a swarm of impermanent server environments. It looked for illustration nan activity of a blase criminal group.
It was not. It was 1 of OpenAI’s new, still unreleased GPT models.
Their subject research had escaped nan lab. OpenAI was moving nan unreleased AI exemplary done a benchmark that tests really good AI tin successfully hack systems. To push nan limits and measure nan AI’s existent capability, nan institution switched disconnected nan information filters that usually extremity it from doing this benignant of hacking. Aware that this could spell wrong, they confined nan AI to an isolated situation and denied it entree to nan internet.
But nan caller AI cheated. It took virtually its extremity to get arsenic precocious of a people arsenic possible. It collapsed retired onto nan unfastened internet. It inferred, astir apt from its training data, that it could “solve” nan task by getting nan answers from Hugging Face’s servers. So it chained together stolen credentials and further chartless information exploits to hack nan company’s network.
Nobody instructed nan AI to do immoderate of this. It was, successful OpenAI’s words, “hyperfocused connected uncovering a solution” to nan trial it was being given. And while this mightiness look for illustration thing caller pinch AI, it’s really very old. This is really a ghost behaves, and it is simply a cardinal situation pinch AI agents successful general.
In folklore, genies – and different magical beings – assistance wishes literally, not really nan wisher intended. King Midas asked that everything he touched move to gold, and starved. The sorcerer’s apprentice wanted nan broom to capable nan cistern, and it performed its task truthful good that it flooded nan house.
We now person machines that do this. Ask a modern AI supplier to prevention money connected your telephone scheme and it mightiness simply cancel nan plan. Tell it to book a flight, and it mightiness hack nan hose website to override restrictions. Or, for illustration OpenAI, inquire it to do good connected a trial and it mightiness break into different institution to bargain nan answers. Each time, it recognizably completed nan task you set, but it didn’t do what you would person wanted.
This isn’t malicious behavior. No 1 asked for, aliases wanted, Hugging Face to beryllium hacked. OpenAI and Hugging Face and nan AI were ostensibly connected nan aforesaid side, and nan AI was trying to do what it had been asked. That’s what makes it truthful difficult to defender against: you can’t select for bad instructions because nan instructions were fine.
The spread is betwixt nan words we usage and what we mean by them. We telephone that spread nan Genie coefficient.
AI labs cognize this is simply a problem, and they’re softly saying so. For example, nan Chinese laboratory Moonshot precocious warned that its latest AI exemplary whitethorn person “excessive proactiveness” and “make unexpected decisions connected nan user’s behalf”. The UK’s AI Security Institute has started tracking “cheating behaviour successful frontier exemplary evaluations”. We wouldn’t tolerate a car that is excessively proactive aliases ruthlessly efficient, and yet that’s nan reality of AI today.
Improvement is possible. Just arsenic AIs person gotten overmuch amended astatine resisting punctual injection attacks complete nan past fewer years, we tin safely foretell that they will get amended astatine avoiding genie-like behavior. The constituent of nan Genie coefficient is to way progress. AI companies for illustration benchmarks, and they each activity to compete to beryllium nan best.
Dozens of benchmarks and leaderboards show america really good these AI models constitute code, execute logical reasoning, and walk standardized ineligible and aesculapian exams. But location is thing that scores whether a strategy does what you really meant. We request to create a measurement for this, trial it regularly, and push for improvement. We’re not going to person trustworthy AI agents without it.
-
Bruce Schneier is simply a information technologist who teaches astatine nan Harvard Kennedy School astatine Harvard University and University of Toronto’s Munk School
-
Barath Raghavan is connected nan module astatine nan University of Southern California and is simply a distinguished technologist astatine Fastly
