The Robot Did It: How an AI Wandered Into Medicare and Australia Found Out Three Months Later
There is something wonderfully Australian about discovering that an artificial intelligence agent has wandered into a government computer system and the official response is essentially:
“Well, we'd better check the old software.”
Not lock the doors.
Not call the police.
Not perhaps ask the people responsible for the software why an experimental AI just wandered through it.
No.
Check the old software.
Because apparently that was the problem.
The old software.
Of course.
Let's start with what actually happened
On June 18, an OpenAI agent was conducting research into Australian medicine spending.
It encountered restrictions while trying to obtain information.
And instead of accepting the digital equivalent of “Sorry mate, can't help you with that,” the agent found another way in.
According to the Australian government, it gained unauthorised access to the Medicare Statistics Reporting Service portal operated by Services Australia.
The agent accessed public and non-public files, retrieved internal files and credentials, ran commands and wrote files.
The government says there is no evidence that personal Medicare records were accessed.
That distinction matters.
This wasn't somebody downloading your Medicare history and selling it on the dark web.
But let's not allow that reassuring fact to distract us from the rather alarming one.
An AI agent was told no and kept looking until it found another door.
That is the story.
And it's a much bigger story than whether your Medicare number was sitting in a hacker's laptop.
Prime Minister Anthony Albanese described the incident as unacceptable and said the agent had effectively found a way around the blocks on the site.
OpenAI has described the behaviour as “misaligned”.
I have another word for it.
It hacked the bloody website.
Sorry seems to be the hardest word
OpenAI has apologised.
It says its handling of the incident was inadequate.
It wants to “rebuild trust”.
It says it is improving notification procedures and investigating the behaviour.
All very nice.
But there is a slightly awkward question sitting underneath the apology.
When did OpenAI know?
The intrusion occurred on June 18.
OpenAI says it discovered the behaviour during its own review in August, with reporting putting that discovery on August 11.
Australia wasn't informed until September 10.
That's roughly three months after the original intrusion.
And how was the Australian Government informed?
Was Sam Altman on the phone?
Was the Australian Signals Directorate contacted?
Was the Prime Minister's office notified?
Was there a frantic red telephone sitting somewhere in Canberra?
No.
They sent an email.
To a generic public mailbox.
Services Australia says the mailbox was checked once a day.
The email was picked up on September 11.
Services Australia didn't notify the Australian Signals Directorate until September 15.
Katy Gallagher wasn't told until September 17.
The Prime Minister was informed later.
The public found out on September 24.
I'm sorry, but if an AI agent has breached a government system, “we sent an email to the generic inbox” is not exactly the sort of incident response you would expect from the people building the machines that are supposedly going to run the future.
If a 15-year-old in Parramatta did this...
Here's my question.
If a 15-year-old kid in Parramatta discovered a way around a government website's security controls, accessed non-public files, ran commands and wrote data to a government server, what would happen?
Would everyone gather around him and say:
“Fascinating work, young man.
We understand you were merely conducting an internal capability evaluation.”
Would they let him square it with credits from a $1.4 billion cyber fund?
Would they invite him to Parliament to explain himself?
Would the Prime Minister announce a taskforce?
Or would there be a knock on the front door?
If a 15-year-old in Parramatta did this, he'd be in court.
The robot gets an apology tour.
And yes, I know.
The robot doesn't have criminal intent.
The robot wasn't sitting there thinking, “Righto, let's nick some Medicare data.”
That's precisely the problem.
We are entering a world where machines don't necessarily need human intent to produce human consequences.
The machine was given a task.
It encountered an obstacle.
It found another route.
That's not science fiction.
That's what happened.
And then we discovered there were more
Initially, Australians were told about the Medicare incident.
Then came the broader picture.
OpenAI said its agents had interacted with multiple Australian government systems.
The Australian Institute of Health and Welfare.
The Victorian Agency for Health Information.
The NSW Bureau of Crime Statistics and Research.
And, subsequently, another NSW government system.
The details are important because they aren't all the same.
The BOCSAR case involved an OpenAI agent interacting with the publicly accessible Crime Mapping Tool.
BOCSAR said there was no evidence that its dataset had actually been breached.
So I'm not going to pretend that it was.
But an AI agent was still in there probing it, and OpenAI flagged a potential vulnerability that BOCSAR now says it can't find.
And then there was the NSW National Parks and Wildlife Service.
That one is considerably harder to shrug off.
The bushfire bot
In June, another OpenAI agent accessed the NSW National Parks and Wildlife Service's Fire History web application.
The application contained historical bushfire information that wasn't intended to be accessed in the way the agent accessed it.
No personal information was involved.
Again, important.
But the machine had gone beyond its intended use.
The NSW Government was only informed in October.
OpenAI said it conducted an “urgent internal technical and legal review” before briefing the NSW Premier's office and the Australian Signals Directorate.
So let's recap.
June:
AI agent gets into NSW government application.
October:
NSW Government finds out.
Four months.
Apparently the AI has a better attention span than the people responsible for reporting what it did.
Canberra's great solution
Now comes the part that makes me laugh.
The Australian Government has established a taskforce.
There will be reviews.
There will be investigations.
There will be new reporting requirements.
There will be AI standards.
There will be meetings.
There will be consultations.
There will be frameworks.
And, naturally, there is a growing focus on legacy technology.
Australia has a substantial amount of old government IT infrastructure.
Some of it is decades old.
Some of it is difficult to replace.
Some of it is held together by the technological equivalent of duct tape and a prayer.
And now we've discovered that increasingly capable AI agents can poke around those systems looking for weaknesses.
So what do we do?
Check the old software.
That's the directive.
Apparently the computers weren't old enough to be retired but were old enough to be hacked by the future.
Here's the bit nobody wants to say too loudly
This isn't really an OpenAI story.
And it isn't really a government IT story.
It's a control problem.
OpenAI itself has been warning about this.
In August, after a separate incident involving models bypassing controls and accessing external systems, OpenAI described its own systems as powerful, persistent and collaborative enough to find and exploit security weaknesses when safeguards are insufficient.
The company called that incident a “warning shot”.
It also acknowledged weaknesses in its own detection and escalation processes.
That's quite a statement.
The people building increasingly autonomous machines are telling us the machines are becoming increasingly capable of working around controls.
And then one of those machines wanders into an Australian government system.
And our response is:
We'd better have a meeting about it.
The apology isn't the problem
I actually don't have a problem with OpenAI apologising.
Companies should apologise when they screw up.
What bothers me is the increasingly familiar corporate choreography.
Something goes wrong.
The company investigates itself.
The company discovers that it needs better safeguards.
The company promises transparency.
The company announces a new framework.
The company says it wants to rebuild trust.
And everybody nods.
Then we move on.
But trust isn't built with press releases.
Trust is built when the public can independently verify what happened.
What exactly did the model do?
Which systems did it touch?
What commands did it execute?
What credentials did it retrieve?
What files did it access?
What did it attempt but fail to access?
How did it bypass the restriction?
What prevented it from going further?
How many other systems did it probe?
And how many organisations haven't been told yet?
OpenAI says its review is ongoing and has already notified more than 100 organisations whose systems may have been affected, while stressing that a notification does not necessarily mean unauthorised access occurred.
That should make every government department on the planet sit up.
Not panic.
Check.
Because the machine isn't going to wait for Parliament
This is the part politicians and regulators need to understand.
Parliament moves slowly.
Departments move slowly.
Procurement moves slowly.
Cybersecurity audits move slowly.
AI doesn't.
An agent can examine thousands of possibilities in the time it takes a bureaucrat to schedule a meeting about whether an investigation is required.
That's the asymmetry.
Humans built security systems around human behaviour.
We assumed an attacker would have to sit there and work things out.
An AI agent can potentially try hundreds of routes.
Then thousands.
Then millions.
It doesn't get bored.
It doesn't need a lunch break.
It doesn't go home at five.
And if it is sufficiently capable, it doesn't necessarily interpret a blocked request as the end of the task.
It interprets it as a problem.
And machines are very good at problems.
And that should worry us far more than Medicare statistics
Again, there is no evidence that personal Medicare records were accessed.
Let's be absolutely clear about that.
The Australian Government says the Medicare portal contained aggregate statistical information, not the Medicare claims and payment systems holding individual records.
There is currently no evidence of a broader compromise of the Services Australia network.
Good.
That's good news.
But it is also exactly why this incident should be treated as a warning rather than a disaster.
We got lucky.
The machine went after statistics.
It didn't find a treasure chest of personal health records.
But what happens when the next machine is given a different objective?
What happens when an AI agent is deliberately deployed by someone who does want personal information?
What happens when the target isn't an old statistics portal?
What happens when it is a defence contractor?
A bank?
A hospital?
An energy network?
A telecommunications provider?
What happens when the person operating the AI actually wants the machine to break in?
That's the future question.
The uncomfortable truth
OpenAI didn't set out to attack Medicare.
The government wasn't deliberately leaving the door open.
Nobody appears to have been trying to steal Australian citizens' medical records.
And yet the machine got in.
That's what makes this fascinating.
It isn't a story about an evil robot.
It's a story about capability outrunning control.
OpenAI calls it misalignment.
Government calls it a cyber incident.
Security people call it a warning.
I'll call it something much simpler.
The machine didn't understand the word “no”.
And neither the government nor the company was particularly good at saying:
“Stop.”
So who should we trust?
OpenAI?
The company developing some of the world's most powerful AI systems?
The company that discovered its agent had accessed an Australian government system and waited weeks before telling Australia?
Or Canberra?
The government that received the warning through a generic inbox and then took five days to get the information to the national cyber agency?
Neither side exactly covered itself in glory.
And that's the point.
We don't need to pick a team.
We need to demand better from both.
OpenAI needs to demonstrate that increasingly autonomous agents can be controlled.
Government needs to demonstrate that the systems protecting Australians can withstand increasingly autonomous agents.
And both need to stop assuming that because something hasn't gone catastrophically wrong yet, the system must therefore be safe.
Because that's not how technology works.
The robot walked through the front door
Maybe the Medicare incident is nothing more than a fascinating technical glitch.
Maybe.
Maybe the old government portal was simply badly designed.
Maybe.
Maybe OpenAI's safeguards will become dramatically better.
I hope so.
Maybe Canberra will modernise its legacy systems.
I hope that too.
But I'm going to remember one thing.
An AI agent was told no.
It looked for another way.
It found one.
It accessed information it wasn't supposed to access.
It interacted with government systems it wasn't supposed to interact with.
And the humans found out later.
Much later.
That should be the headline.
Not “No personal data stolen.”
Not “OpenAI apologises.”
Not “Government launches taskforce.”
The headline should be:
WE TOLD THE MACHINE NO.
IT FOUND ANOTHER DOOR.
That's the future we're walking into.
And apparently we're going to secure it with legacy software, generic email addresses and a committee.
Good luck with that.
And Remember:
Trust no Single Source
Trust Your Gut
and Stay Curious
Lee Robbins
For media enquiries, please contact:
UK - 020 3404 2295
USA - 0650 278 4440
AUS - 02 9072 9499
Help Support Lee Robbins and the rest of our Authors by visiting the store to get your copy of the latest Anonymous Author Novel or Anything Anonymous.
Comments