Last quarter, a partner at a mid-size law firm asked one of her associates to summarize a draft acquisition agreement. The associate did exactly that — pasted the full 47-page document into ChatGPT, got a clean summary in 90 seconds, and moved on. Nobody flagged it. Nobody knew. The document contained material non-public information about a publicly traded company.
This isn't a story about a bad employee. It's a story about a governance gap that exists at nearly every professional services firm right now — and most firms don't know the gap is there until something forces the question.
The question nobody asks before clicking "Accept"
When your associate opens ChatGPT and pastes in client data, the answer to "where does this go?" depends entirely on which product they're using and whether your firm has the right agreements in place.
ChatGPT Free and Plus: By default, conversations may be used to improve OpenAI's models. Users can opt out in settings — but that setting is off by default and requires a deliberate action most users have never taken.
ChatGPT Enterprise: Inputs are not used for training, and OpenAI offers a Data Processing Agreement. This is the version regulated firms should be using if they use OpenAI at all — but it requires a paid enterprise license and explicit configuration. Most associates are using the free or Plus tier from a personal account.
Microsoft Copilot (M365): Microsoft does not use your tenant's data to train foundation models. Your data stays within your Microsoft 365 tenant boundary. But this is only true with the right license and only if Copilot has been properly configured — sensitivity labels, permission boundaries, and data classification need to be in place, or Copilot can surface data across the organization that users were never meant to see.
Why this matters for regulated firms specifically
For a firm under Reg S-P, HIPAA, or ABA Model Rule 1.6, the question isn't just about training data. It's about whether client data left your governed environment at all.
Under Reg S-P: The Safeguards Rule requires firms to protect customer financial information. Using an unvetted AI tool that processes client data without a proper data processing agreement is a potential safeguards violation — even if nothing bad happens as a result.
Under HIPAA: Any AI tool that processes Protected Health Information requires a Business Associate Agreement. OpenAI's consumer products do not offer BAAs. Using them with patient data is a HIPAA violation on day one, regardless of intent.
Under ABA Rule 1.6: Attorneys have a duty to make reasonable efforts to prevent the disclosure of confidential client information. The "reasonable efforts" standard now has a technical component — and pasting privileged communications into an uncontrolled AI tool almost certainly doesn't meet it.
The right answer isn't a ban
Prohibiting AI tools entirely doesn't work. Staff will use them anyway, from personal accounts, on personal devices, outside any visibility your IT team has. A blanket ban creates the worst possible outcome: the behavior continues, but now it's hidden.
The right answer is a governed adoption plan. Approved tools with the right agreements in place. Data Loss Prevention configured to flag regulated data in AI prompts. Sensitivity labels on documents so Copilot knows what it can and cannot surface. A written AI acceptable use policy that staff have actually read and signed. And ideally, a named owner for the AI governance program — someone whose job it is to stay current as the tools evolve.
An approved AI tools list with corresponding Data Processing Agreements or BAAs on file.
DLP policies that detect regulated data categories in AI prompts and either block or alert.
Microsoft 365 sensitivity labels applied to confidential and regulated documents — so Copilot respects the boundaries you've set.
A written AI Acceptable Use Policy reviewed annually and signed by all staff.
A named owner — vCISO or vCCO — responsible for keeping the governance program current as the tools change.
The AI era isn't coming. It's here. Every firm in your competitive set is navigating the same question. The ones who build the governance layer now won't just be protected — they'll be able to use AI more aggressively than their competitors, because they'll have the architecture to do it safely.
The ones who wait will be explaining a governance failure to a regulator who has very little patience for "we didn't realize the tool was doing that."