AI systems that stay behind your firewall
How to provide employees with helpful AI inside company boundaries without leaking sensitive data. - ai - security - compliance
In nearly every finance-related organization I’ve encountered, a common pattern
emerges:
Official policy states: “No ChatGPT, no external AI tools - compliance forbids
it.”
The reality is that employees use these tools anyway, often in silence, due to
overwhelming workloads.
This isn’t merely about “productivity hacks.” Employees are sharing parts of
contracts, project documents, internal emails, and even client data with
third-party tools operated by companies outside their own infrastructure and
sometimes outside the EU.
From a compliance perspective, this poses significant risks:
- Data leaves the company’s network and jurisdiction
- There is typically no data processing agreement (DPA) in place, violating GDPR
requirements - There is often no clear explanation of where this data is stored, how it is
utilized, or who may access it - No audit trail exists for compliance reviews or incident investigations
On paper, many organizations are prohibiting this behavior. In practice, they
turn a blind eye, hoping for the best.
Whether we like it or not, employees will continue to use these tools. If they
don’t, in certain roles, they simply cannot keep pace with colleagues who do.
Thus, the real question for companies isn’t: “AI or no AI?”
It’s about choosing between:
- Unmanaged shadow use of public tools with sensitive data
- A clear, compliant, internal method for utilizing AI in the workplace
Both scenarios stem from the same desire for assistance and automation, but the
latter involves:
- Operating on infrastructure controlled by the company
- Defining rules for what data can be used
- Implementing logging, access control, and auditability
- Integrating with real workflows (invoices, contracts, tickets, etc.)
That is what I mean by engineering AI systems behind the firewall.
If you do not want people pasting sensitive data into random web UIs, you have
to give them a fast, useful, and compliant way to use AI inside your own
boundary.
The choice is clear: provide a secure, internal solution, or watch your data
leak through shadow IT while your compliance team documents violations they
cannot prevent.
For a deep dive into the architecture and hardware considerations for building
AI systems behind the firewall, see
“You Need More Than a Big LLM”,
which explores the three-layer architecture pattern and hardware selection
strategies.