Financial services & AI governance
Kevin O’Brien, Head of AI · Video · 6 min read · Updated 2 November 2026
The short answer
Watch: How do you keep AI under control in financial services?
Kevin O’Brien, CSM’s Head of AI, explains the topic. Read the expanded guidance and practical questions below.
What does keeping AI “under control” mean?
It means defining what the system may read, infer, recommend and do, then enforcing those limits throughout the workflow. A model can produce a useful interpretation without being authorised to make a customer decision, change a record or send a communication. Those are separate permissions and should be designed separately.
Kevin’s point is practical: give the model a small, well-defined job and build the necessary software around it. At CSM, we would ask which parts can be expressed as explicit rules, which require interpretation and which remain accountable human decisions. A longer prompt cannot replace that division of responsibility.
Why is a personal chatbot different from a production system?
In an individual conversation, a person can spot an error, ask a follow-up and choose not to use the response. A background workflow may process many cases before anyone sees the output. A small recurring error can therefore accumulate, while a rare failure may have serious consequences in a particular case.
The relevant unit of assessment is the whole process. A correct summary can still be sent to the wrong person. A valid-looking field can still contain the wrong amount. A reliable model does not compensate for excessive system permissions, stale source data or a missing approval step.
Which responsibilities should stay in ordinary software?
Keep authentication, access permissions, required-field checks, approved calculations, routing rules and release gates in controlled software wherever they can be specified explicitly. A language model may extract or classify information, but its proposed result should be checked before it becomes an operational instruction.
For example, require an output to contain permitted field names and valid values. Check monetary totals using defined calculations, not generated prose. Confirm that a referenced record exists and belongs to the relevant customer. These checks catch particular error classes; a well-formed output can still be factually wrong, so source verification remains necessary.
What could this look like in financial services?
Consider an illustrative internal document-intake workflow. A customer sends supporting documents. Software checks the case identity and file requirements. AI proposes document categories and extracts relevant passages. Validation checks completeness and inconsistencies. A reviewer sees the originals beside the proposed fields and resolves exceptions before an approved next step.
This example does not authorise the model to approve credit, determine eligibility or release funds. A useful first scope may stop at preparing the case for a qualified person. The design must be assessed for the specific product, customers and jurisdiction; a workflow that is suitable for internal preparation may be unsuitable for an automated customer decision.
Is putting a human in the loop enough?
No. Review needs a defined purpose, usable evidence, sufficient time and the authority to change or stop the result. If the person only clicks an approval button without understanding the underlying case, the organisation has not established a dependable challenge process.
Decide what triggers mandatory review, what the reviewer must check and what happens when the queue exceeds capacity. Track overrides and missed issues. For high-consequence decisions, a low average error rate is not a reason to remove oversight without a specific assessment. The FSB’s October 2026 sound practices encourage oversight proportionate to the materiality, risk, autonomy and complexity of the use case.
What regulatory questions should an Irish or EU firm ask?
Identify the obligations that apply to the actual use, including data protection, sector requirements and any relevant AI Act classification. Do not assume that using a familiar model, adding a reviewer or buying certified cloud infrastructure makes the finished workflow compliant. The firm’s compliance and legal functions should assess the use case and required evidence before launch.
Under GDPR Article 22, particular protections concern decisions based solely on automated processing that produce legal or similarly significant effects. Exceptions and safeguards require careful assessment. The Irish Data Protection Commission’s Article 22 guidance explains the scope and relevant rights. This is not a blanket prohibition on all AI assistance, nor permission for every workflow that includes a nominal reviewer.
The Central Bank of Ireland’s supervisory outlook examines AI-related consumer protection and operational resilience risks. International sound practices can inform a design, but they should not be presented as a substitute for applicable Irish or EU requirements. This article provides an operational approach, not a compliance determination for a particular firm.
How do you prevent the model from doing too much?
Expose only the functions and data needed for the task. Separate read access from write access. Require independent authorisation for consequential actions, and validate the requested action against the current case and the user’s permissions. Do not let text in an uploaded document grant authority to perform an action.
OWASP describes the risk of systems giving models excessive functionality, permissions or autonomy in its Excessive Agency guidance. Our practical recommendation is to make the system’s allowed actions narrow enough that a misleading answer cannot immediately become an unrestricted business action.
What should be tested and recorded?
Test missing documents, contradictory instructions, altered permissions, malformed outputs, repeated requests and failed connections. Include realistic attempts to manipulate the system. Check that retries cannot issue the same action twice and that a partial failure leaves a visible, recoverable case. These are properties of the software as well as the model.
Keep an appropriate record of source references, system and model versions, validation results, approvals and actions. Avoid logging unnecessary sensitive content. Agree retention and access to the audit records. The FSB also recommends clear accountability, risk assessment, performance monitoring and management of third-party dependencies; the objective is evidence of control across the lifecycle.
How would CSM begin?
We would choose one bounded workflow, identify a business owner and involve the people responsible for operations, risk, data and technology. Agree acceptance criteria, prohibited actions, escalation routes, change control and support. Run the workflow in a controlled setting before expanding responsibility.
Success means useful work completed within an understood boundary. The question is not how much authority the model can be given. It is how much value the business can obtain while retaining the evidence and control needed for the decisions it owns.
What should I read next?
Explore how to choose workflows for AI automation and which leadership responsibilities should remain human.
Questions for your leadership team
What is the model permitted to decide?
List its exact task, forbidden actions and the authority retained by software and people.
What evidence will a reviewer see?
Make the original source, proposed output and validation failures available together.
Which obligations apply to this use case?
Have the relevant compliance, legal and privacy owners assess the actual product, customers and jurisdiction.
Can we stop and reconstruct an action?
Test the pause mechanism, audit trail, rollback or recovery process and customer support route.
What changes trigger a new assessment?
Include model updates, source changes, new customer groups, wider permissions and increased autonomy.
Full video transcript
Transcript reproduced from the supplied recording script. The expanded guidance distinguishes the spoken examples from current implementation recommendations.
Hi, everyone. Rory Kieran here from CSM. So in this part two of our conversation with Kevin O’Brien, our head of AI, we’re gonna be talking around use cases. I think, you know, AI is such a broad topic, but actually talking through some of the key problems, specifically in certain industries.
We’re gonna have an emphasis on financial services because Kevin has spent a lot of his career over the last ten years working in some very large financial services companies. Think I regulation, compliance is really important in terms of talking about that topic because I think it’s what scares a lot of financial service companies when when they think about AI and taking things out of their control. We’re gonna talk around setting up control mechanisms, putting structure around AI, and just loads of different use cases. So maybe just to start things off, Kev, obviously, you have worked in a lot of financial service companies, but we might just talk around the importance of compliance and regulation.
And when it comes to AI, obviously, I think there’s there’s a level of nervousness around control and structure around it. And what me and you have talked about, you know, quite a lot is there’s all these AI agencies that are setting up now because it’s become such a hot topic, but it’s really, you know, younger people who might have watched a couple of YouTube videos and and learned a bit about AI, and and and now they’re calling themselves an AI specialist. And, you know, I have to, I suppose, lean into that because it it stressed the importance of there’s a massive difference between someone who’s been in the space for ten years before it was a a hot topic and someone who’s just starting out.
Could you maybe just give the viewers a bit of an understanding as to…
And and more so stress the importance of setting up those control mechanisms and using software in the right way to sort of protect yourself and not leaning too much into an LLM. I think it’s a really important part to maybe start with.
Yeah.
Yeah. So I think, obviously, LLMs are incredible. They can do amazing things, and they become so powerful, they’re just being used everywhere. But I do see it’s becoming quite a problem how much people are trusting LLMs.
And, yeah, they they can do amazing things with the right setup, but they can also make crazy mistakes that you just…
You look at them and think how how could it even make that mistake. I think there’s…
There used to be a meme going around of everyone asking chat GPT how many r’s are in the word strawberry, and I could never get it right because that’s just not how these models work. And they’ve actually fixed it now just because it’s a meme. They’ve made us give the right answer. But it’s just a good example of, like, how can something be so smart and know everything about the world, but it can’t even count how many letters are in a word?
So, yeah, I think working in in the industry for quite some time now, I think I’ve I’ve built up, I guess, like, a a triangle of skills that’s becoming more and more rare. One is obviously the AI side of things. One is just the software engineering that’s used to build everything around the AI.
Well, I think that’s one area that I’d like to unpack a little bit more. So when it comes to structuring that, we talked a lot around the importance of setting up control mechanisms using software. So I think these new AI agencies that are forming, they don’t probably have that same level of depth in terms of the software background and the data background that that you do. So when coming into financial services, what would be that sort of layer of structure that you would typically recommend by using software to sort of control what what the steps are? That’s the best way I’m explaining it, but I know it’s probably.
Yeah. So I think, like I said, there’s just become a big trust in LLMs, which I think for a lot of cases is okay, but for heavy regulated cases, you really need to, what I do anyway, is give as little responsibility to the AI as possible. So building these systems, there there is a lot of work into building just the the foundation of software around the AI and around the LLMs. And, yeah, you see projects now where it’s really about just giving the LLM a massive prompt with all the information and just hoping it outputs the right thing and just trusting that and using that in production then as a as a trusted system.
Whereas the way I build it is actually the opposite, where all the problems I put into LLMs are usually very, very small because I built so much guardrails and control in the software around it that, I’ve taken as much responsibility that can be, like, determined by the software on the side of the software than anything that actually requires the LLM to make a decision that can’t be done directly in software. Only that one most, minute decision that can be made by the LLM, I then pass that off and then process it again in the software afterwards. So it’s really about just, yeah, taking as much responsibility away from the AI and just controlling as much as you can, which I think people don’t realize the importance of that. And, yeah, it just… It sometimes can end badly when these models do something that you’re not expecting or something you haven’t handled.
And, yeah, it’s… If you look at people on a personal level using ChatGPT, it’s fine there. When you’re looking at the screen and you’re asking it to do something, and you can change your prompt and adjust it or follow-up and and correct something if it makes a mistake. Whereas if you’re building an AI system that’s working in the background in the cloud or whatever on your website or your app or wherever it might be, and it might be processing requests for hundreds or even thousands of customers every single day, obviously can’t have a person overlooking every single thing that AI does and and being able to correct it. So it just goes to show how important it is to build, like, really robust software and systems around the AI that’s making these decisions.
Put these questions to work in your business
Bring a real decision or workflow to a conversation with CSM. We can explore the evidence, responsibilities and practical steps needed to make AI useful for your team.