How 2iG Uses AI Agents to Guarantee Data Feed Quality

At 2iG Solutions, our analytics platforms live and die by nightly data feeds. Every night, we ingest policy, loan, and quote data from our customers’ systems — loan servicing platforms, rating engines, policy admin systems — transform it, validate it, and load it into the data warehouse that powers their dashboards the next morning.

Anyone who has run data integrations knows the uncomfortable truth: the pipeline usually works. And “usually” is the problem. A feed can succeed while silently dropping records. A validation rule can flag something scary that’s actually routine. The traditional answer is a wall of alert emails that a human has to read, interpret, and investigate — every single day, forever.

We just replaced that with something better: an AI agent that does the investigation itself.

How it works

Every morning, after the nightly data jobs finish, a scheduled AI agent (built on Anthropic’s Claude) wakes up and does what a good data engineer would do on their first hour of the day:

1. It checks every job. Load ledgers, error logs, rejected-record files, pipeline execution history — across both our staging and production environments.
2. It separates signal from noise. The agent works from a runbook we maintain in our codebase: a living document that catalogues what “normal” looks like, which warnings are known-benign, and what the signatures of real failures are. When we learn something new, we update the runbook — and the agent is smarter the next morning.
3. It investigates anomalies like an engineer, not a monitor. This is the part that still surprises me. When something looks wrong, the agent doesn’t just report the symptom. It reads our actual source code, queries the warehouse (read-only), compares against historical patterns, and forms a root-cause hypothesis with evidence.
4. It emails me a verdict. One digest per environment, every morning. The subject line tells me instantly: ✅ all clear, or ⚠️ N issues. The body leads with a summary and a proposed fix, with the detail underneath if I want it.
5. A human makes every change. The agent never remediates anything on its own. If it finds a real issue, I open a session with it, review its analysis, and tell it to prepare the fix — which still goes through our normal branch, test, and release process.

Did it actually find anything?

On its very first run, the agent caught a data-materialization step that had been silently failing for two nights — the job reported success, but one downstream metric wasn’t being populated. No error email had fired, because the failure was swallowed inside a batch that otherwise succeeded. The agent noticed the gap, traced it to a specific line of SQL in our code, and proposed the exact fix. It was right.

That’s the kind of issue no threshold-based alert would ever have found — because nothing “failed.”

The part the security-minded reader is waiting for

Giving an AI access to production data infrastructure should raise eyebrows, so we designed for least privilege from day one:

• The agent uses dedicated, read-only credentials — a separate identity with an explicit allow-list, and a permissions boundary that denies everything else. It can look; it cannot touch.
• Its database access is SELECT-only, verified by actually attempting writes and confirming they’re denied.
• A tripwire alerts us if the agent’s identity ever attempts anything outside its allowed scope.
• And the daily digest doubles as a dead-man’s switch: I get an email even when everything is clear, so silence itself is a signal.

Why this matters for our customers

Data quality is the foundation of everything we sell. A dashboard is only as trustworthy as last night’s load. What this system gives our customers is a standard that’s hard to hit with alerting alone: every feed, every night, reviewed with engineering-level scrutiny — with a human accountable for every change.

The deeper shift is this: monitoring tells you that something happened. This tells you what happened, why, and what to do about it — before the workday starts.

We think this pattern — AI agents doing genuine investigative work inside tight, auditable guardrails — is where operational AI is actually useful today, well beyond chatbots. We’ll keep sharing what we learn.

Harry Snyder is the co-founder and CTO of 2iG Solutions, which builds analytics platforms for premium finance companies, MGAs, and carriers.

Originally published on LinkedIn — join the discussion there.

Learn more about PFC Insight, MGA Insight, or contact our team.