AI, without the technology lecture.
AI isn't one product, one model, or a sequence of seven maturity stages. It's a broad field that now includes capabilities with very different strengths and limitations.
Machine learning finds patterns in data, supporting forecasting, classification, and anomaly detection. Generative AI is a branch of machine learning that produces new outputs, such as text, code, images, or audio. Large language models are one type of generative AI, especially useful for language-intensive work.
Retrieval-augmented generation, or RAG, is a technique for supplying relevant retrieved information to a model. It may improve grounding, but the answer still needs evaluation. An agent is a system design: a model combined with instructions, tools, application logic, and permissioned actions. Agents vary widely in autonomy. Calling something an agent tells us little about whether it should be trusted with a particular decision.
Three practical executive questions
The airport problem:
busy isn't the same as arriving.
An airport can be full of activity without getting travelers to their destination. The same is true of an AI program: licenses, experiments, demonstrations, and even deployed agents are evidence of activity, not proof of transformation.
Experimentation is essential. A bounded pilot can tell us whether a use case is feasible, how reliable a model is, where employees encounter friction, and what new risks appear. The mistake is treating a successful demonstration as the finish line.
To move from a pilot into everyday operations, somebody must redesign the workflow, establish ownership, evaluate the quality of decisions, and demonstrate that customers or employees are actually better served. Some experiments deserve to stop. Some need a narrower scope. Others uncover a process problem that AI wasn't needed to fix.
The CFO's question:
What changed?
A useful AI business case should survive a conversation about the before-state, the after-state, and everything it costs to operate the solution.
I would establish a baseline before deployment: work volume, elapsed time, labor effort, error and rework rates, service levels, and customer impact where relevant. Then measure whether the new way of working changed those results. Usage matters because a capability no one uses has limited value. But token consumption and active users do not establish financial return.
Value can come through growth, capacity, cost, quality, customer and employee experience, or reduced risk. Not every outcome has a dollar figure, and not every use case should be judged by the same metric. What matters is credible evidence of change.
Investment and use
Licenses, consumption, adoption, and cost.
Quality of work
Cycle time, accuracy, review effort, rework, and exceptions.
Sustained improvement
Throughput, SLA, backlog, quality, and capacity.
Realized outcome
Cost, growth, service experience, resilience, or reduced exposure.
A small example shows the distinction.
Imagine a team handles 1,000 routine service requests per month. The average handling time falls from ten minutes to four. The arithmetic is straightforward.
Illustrative calculation only. Not an actual deployment result or financial saving.
Those 100 hours are potential capacity, not 100 hours of payroll reduction. Was capacity redeployed to reduce backlog? Did a team avoid overtime or increase throughput? Did the service actually improve? What were the costs of inference, integration, evaluations, human review, rework, monitoring, security, and support? A faster task that produces more downstream errors is not necessarily a better process.
For me, the financial story becomes credible when operational owners can show where the benefit landed, what it cost to sustain, and whether it still holds six months later. That's the difference between a modeled benefit and a realized result.
Reinvent the process.
Don't just accelerate a task.
A ten-second improvement to one handoff is not valuable if the customer still waits three days at the next one. I look at the entire flow of work before deciding which technology belongs in it.
Start with the outcome. Map the workflow, the exceptions, the data dependencies, and the points where people make judgments. Eliminate unnecessary steps where you can. Then select the smallest combination of existing systems, traditional automation, AI assistance, and bounded agents that can deliver the result reliably.
A routine reminder, an approval threshold, or a scheduled status update might be better handled by a deterministic workflow. Language-intensive interpretation may justify an LLM with human review. A tool-using agent might help with multi-step work, but it needs explicit permissions, monitoring, and a clear boundary around what it is allowed to change.
Rules + workflows
Known inputs, defined thresholds, predictable steps.
AI-assisted judgment
Interpret language, synthesize context, prepare a recommendation.
Tool-using agents
Coordinate bounded actions across approved tools, with oversight.
Go-live is the beginning
of accountability.
An AI capability deployed into an enterprise workflow becomes part of the operating environment. It must be owned, supported, evaluated, secured, and improved.
Data changes. Models and suppliers change. An upstream system gets a new field. A new group of users arrives with cases the original test set never covered. If no one owns the capability after launch, today's impressive pilot can become tomorrow's operational incident.
I'd expect clear ownership on three fronts: a business owner accountable for the outcome, a technical or service owner accountable for production reliability, and the appropriate risk and control functions accountable for their oversight. Those roles need to work together, not pass responsibility from one committee to the next.
Operational discipline includes identity and access, approved knowledge sources, integration health, model and version changes, evaluations, quality thresholds, cost controls, monitoring, incident response, and a feedback channel from the people using the system. None of this requires waiting for perfect data before learning. It does require designing the feedback loop from the outset.
outcomeReview and repeat
Choose an outcome and baseline.
Bound the scope and acceptance tests.
Capture performance, quality, risk, cost.
Investigate feedback and failure modes.
Retest, release, and monitor adjustments.
Validate results and restart the loop.
Govern the risk.
Don't govern the curiosity out.
Governance works when it is proportionate to what the system can do, what it can access, and what happens if it gets something wrong.
There is a meaningful difference between drafting an internal summary, suggesting a decision, and updating a customer's record. Treating all three as identical risks creates either an unnecessary approval queue or dangerous gaps. Neither is a useful operating model.
I favor minimum viable governance: clear boundaries for low-risk experimentation, increasing controls as consequence and delegated authority rise, and the ability to strengthen those controls as evidence emerges. This is a management discipline, not a one-time compliance checklist.
INFORM
Draft, research, summarize. Use approved tools, appropriate data boundaries, and human review.
RECOMMEND
Prioritize, classify, propose actions. Add evaluation criteria, quality checks, and accountable approval.
ACT
Update systems or records. Require least privilege, logs, authorization, escalation, and recovery where appropriate.
Cross-cutting controls include supplier and model review, confidential data handling, access permissions, test coverage, auditability, incident procedures, and a way to stop or roll back actions when appropriate. The NIST AI Risk Management Framework and ISO/IEC 42001 are useful reference points for shaping that capability. Mentioning them isn't a claim of certification or legal compliance.
Good governance gives employees confidence about what is safe to try, leadership visibility into what is working, and customers reasons to trust the result.
More output isn't
the same as better work.
The developer jokes write themselves. The interesting question isn't whether AI can produce code or documents faster. It's who understands the implications, notices what is wrong, and accepts responsibility for the outcome.
AI can compress repetitive work and make useful knowledge more accessible. It can also create a new burden: checking plausible outputs, understanding hidden dependencies, and deciding when to trust, reject, or escalate a recommendation.
That makes domain expertise, architecture, testing, communication, coaching, and responsible judgment more important, not less. People need to understand the whole process, not just become efficient at issuing prompts.
I also worry about the development path for newer professionals. If we automate the work through which people used to learn, we need to deliberately design new ways to build experience. Otherwise, we may gain short-term speed while weakening tomorrow's talent pipeline.
The goal isn't to count how often employees ask AI for help. It's to equip them to do better work, challenge bad answers, and own decisions that matter.
Questions worth
putting on the table.
These are the questions I would want the business, technology, finance, and risk leaders to answer together. No maturity score required.
- What business problem are we changing, and why does it matter now?
- What does the baseline look like, and what would count as success?
- Which part truly needs AI rather than a simpler automation?
- Who owns the outcome and the service after go-live?
- What information and actions can this capability access?
- How will we detect errors, harm, rework, or unacceptable cost?
- What will we actually do with any capacity we release?
- How will we know it's still delivering value six months later?