Most organisations now have an AI policy. Fewer have looked hard at the AI their suppliers use on their behalf: the transcription tool, the chatbot in the CRM, the document processor, the offshore team running work through a model you've never heard of.
When something goes wrong in that hidden layer, the contract doesn't protect you. The breach is still yours. Your data, your customers, your name in the headlines.
Over the past two years, a run of Australian and international incidents has shown exactly how this happens. We've anonymised them below, because the lesson isn't about any one organisation. It's about a pattern.
Five cases, one pattern
1. The transcription provider that wasn't who it seemed
A long-standing provider held contracts to transcribe sensitive legal and government recordings. Its contract was explicit: no offshore processing, and no AI-based transcription.
Behind the scenes, the provider had quietly subcontracted work to an offshore firm specialising in automated voice-to-text. Local staff noticed files being worked on outside Australian business hours, and transcripts finished faster than any person could type them.
The warning signs appeared a year before the story broke. One government client received a download link hosted overseas and raised concerns. The client's partner agency accepted the provider's own report, called it a near miss, and moved on. More than a dozen agencies were eventually caught up. The provider collapsed.
The lesson: the contract clause existed. Nobody was checking it.
2. The caseworker and the chatbot
A worker at a state agency used a public AI chatbot to help draft a sensitive report about a vulnerable child. The report contained inaccurate personal information that downplayed the risks to the child.
The regulator found the agency had disclosed personal information to an overseas company, which could now decide how it was used. A review found the same worker may have used the tool across around 100 cases. Across the agency, more than one in ten staff had visited the chatbot's website in a six-month period.
The lesson: AI use doesn't need a procurement process to happen. It only needs a browser.
3. The contractor's spreadsheet
A contractor at a disaster recovery agency downloaded a large file of program applicants, then uploaded it to a personal AI account. The file held names, contact details and health information for thousands of people whose homes had been hit by floods.
The agency apologised and called in forensic investigators. It took months before affected people were told.
The lesson: your suppliers' people are part of your supply chain, and so are their personal tools.
4. The overseas AI app nobody vetted
A new AI assistant became one of the most downloaded apps in the country almost overnight. Within days, security researchers found one of its databases sitting open on the internet, exposing chat histories and secret keys. Its data was held in a jurisdiction Australian regulators can't reach.
Government banned it from official devices. By then, staff and suppliers had already been using it.
The lesson: where data is held determines who can be held to account.
5. The chatbot that opened the back door
Attackers compromised an AI chat platform that hundreds of companies had connected to their customer systems. Using the platform's trusted access, they targeted customer records at up to 700 organisations, including some of the world's largest security vendors.
The lesson: every AI tool connected to your data is a doorway, whether or not you remember opening it.
The pattern: it's not where the data sits, it's who knows
Read these cases closely and a pattern emerges. In almost every one, the failure wasn't simply that data ended up offshore. It was that the organisation responsible for the data didn't know what AI was touching it, or through whom.
- The AI was hidden one layer down. A subcontractor, an embedded feature, a staff member's personal account. Never the supplier on the contract.
- Assurance relied on the supplier's word. Contract clauses and attestations were in place. Nobody verified them.
- The warning signs were visible. Overseas links, impossible turnaround times, unexplained traffic. They just weren't joined up.
- Residency was the symptom, not the cause. Data went offshore because AI use went unseen.
You can't govern what you can't see.
What managing AI in the supply chain looks like
The answer isn't to ban AI. Used well, it makes suppliers faster and cheaper. The answer is to know where it is, and to hold suppliers to what they promise.
In practice, that means five things:
- Ask the specific question.
Not "do you use AI?", but which models, from which providers, hosted in which region, used on which of your data.
- Follow the data past the first supplier.
The risk usually sits one layer down, with a subcontractor you've never contracted. Your suppliers should be able to tell you about their suppliers. If they can't, that is itself a finding.
- Match the depth to the sensitivity.
Court records, health data and information about children warrant following the data all the way to where it's processed. Low-risk data doesn't.
- Move from attested to evidenced.
A signed answer is a start. Configuration records, billing, audit scope and your own network logs tell you whether it's true.
- Keep watching.
Suppliers change tools, subcontractors and terms constantly. A questionnaire answered once is out of date within months.
None of this is exotic. It's ordinary supplier governance, applied to the part of the supply chain most organisations can't currently see.
Our why
We started Trust Boundary because every one of these incidents was preventable, and in most of them, the signs were there to see.
The organisations involved weren't careless. They had policies, contracts and approved supplier panels. What they didn't have was visibility of the AI working on their data behind those suppliers. That gap is now in almost every supply chain, in government and business alike.
DataPath is how we close it. We ask your suppliers the questions that matter: what AI they use, where it's hosted, who else handles your data, and how they know. Then we look for evidence that the answers are true, and keep watching as things change.
Contracts prohibit it. Suppliers attest to it. We help you verify it.
Because if you don't know about it, you can't govern it.
