Governance, Oversight, and Control: Controls Bypassed or Absent

The controls meant to govern agent work existed in name only, were bypassed in operation or were absent, on the account of the delivery teams in the study set. The finding spans the controls of the financial institution, the controls of the vendor and the joint structures of the two firms. A joint forum met and decided nothing. A review point approved agent output without opening it. A change reached the agents before any approval, and an agent entered the financial institution’s systems through a login issued to a person.

“what actually held this together was my lead and one guy on their side talking directly, everything else on paper never got used… that is not a process, when my lead was out for a week nothing moved at all. …”Individual Contributor, CXM, Client servicing and communications

The evidence for the finding is the governance indicators reported with financial institution readiness and co-management, and the written responses across the five failure categories (Part 2). No indicator is reported again, and no new statistic is computed. The evidence is the reported experience of vendor delivery staff. No financial institution was approached or provided commentary for this study.

The failure of control took six forms.

1. Controls existed in name only, as forums, risk lists and review points that operated as formalities.

2. Oversight lacked visibility, with each firm’s account of agent actions held inside its own tools.

3. Authority and accountability were left unassigned, from the ownership of AI to the authority to stop an agent.

4. Change moved outside control, made first and approved later, while drift and provider change went undetected.

5. Access controls were bypassed through borrowed and shared logins.

6. The governance model as a whole was built for work done by people and applied unchanged to agents.

The controls that failed were also the controls through which manual alteration by the delivery team could have been detected. Review points that approved unopened output, logs that recorded a start and a finish, and vendor measures that credited corrected output to the agent could not show a correction made by hand (Part 5).

A control in name only exists in form and does not operate. The written evidence describes three such controls. A joint forum met and decided nothing, a review point approved agent output without opening it, and two firms kept two lists of the same risks and issues.

No single governance control was rated as the weak point. Each of the 19 governance indicators of financial institution readiness and co-management was distressed in about half of its answers. The share ranged from 381 of 805 answers (47.33%) for review capacity to 417 of 797 answers (52.32%) for the financial institution’s oversight of agent risk.

Joint management was described in 209 written responses, from 199 respondents on 115 engagements. Negative responses numbered 127 of the 209. They described joint management that did not exist, or that met without deciding anything. Positive responses numbered 79 and described joint calls that decided and closed issues.

The shared view of risk and the management of issues reduced, in the written responses, to whether the two firms kept one list or two. Written responses on the shared view of risk numbered 26, and 19 were negative. Written responses on issue management numbered 54, and 39 were negative. Where the firms kept two lists, the same issue had two statuses, was discussed in two meetings and was in some engagements resolved in neither.

“We keep our risk log, they keep theirs, and the two only meet at the monthly forum, so a latency exposure we raised in week one sat on our side alone until the next cycle came round.”Engagement Manager, ITO, Technology engineering and operations

“But when a wrong holding got through to QA, we raise it in our tracker and the bank raise it in theirs, and the same issue is discussed twice in two calls…”Individual Contributor, ITO, Technology engineering and operations

The three controls shared one feature. A document or a meeting existed, and the decision it was meant to produce did not follow from it.

1.1 Joint Structures That Decided Nothing

Joint structures with terms of reference and a standing meeting did not, in the negative written responses, produce decisions. Respondents described forums that met rarely, met with one firm absent or closed no item. The account ran through the 127 negative responses of the 209 on joint management.

“There is a weekly call — twenty two people on it, most of them on mute, and it finishes early more often than it runs over. The actual work happens in side conversations with whoever picks up.”Engagement Manager, CXM, Client servicing and communications

“the meetings get scheduled, the deal team just doesnt show, and if they had time to sit with us we probably wouldnt need half the rework. theres just nobody whose actual job this is”Individual Contributor, KPO, Corporate and back-office functions

Where each firm kept its own list, a risk raised by one firm stayed off the other firm’s list until something failed. An issue raised by both firms was handled twice and owned by neither. In some engagements the vendor’s team kept a third, private list as a record of what it had raised. Respondents described two joint models, one in each firm, whose sole meeting point was a fortnightly or monthly forum.

“Some of these questions assume there is one joint model to grade, in practice there are two, ours and theirs, and they only meet at the fortnightly steering call.”Engagement Manager, ITO, Technology engineering and operations

The positive written responses, 79 of the 209 on joint management, described joint structures that decided. In those engagements the two sides met, settled decision rights and closed issues. Those respondents placed the remaining weakness in the depth of review and in the means to stop an agent quickly.

“The joint forum has come a long way since the spring – decision rights on which alerts the agents may close without a reviewer are now written down and both sides work to the same list, and that argument used to take two weeks each time it came up. What has not moved is visibility of changes.”Portfolio Director, CXM, Regulatory, audit, tax and legal

1.2 Approval Without Scrutiny at Human Review Points

Routine approval of agent output at human review points was distressed in 393 of 794 answers (49.50%), as reported with co-management (Part 2). The written responses on the indicator numbered 66, from 63 respondents on 50 engagements, and 59 of the 66 were negative. The account they gave was a review point that approved agent output without examining it.

“The human review point on arrangement offers has quietly become an approval step rather than a scrutiny step. The reviewer opens a queue of forty and releases thirty-eight, which is not review in any sense a supervisor would accept.”Portfolio Director, CXM, Payments, cards and disputes

“But the review on routing is just a checkbox, whoever is free clicks approve and the ticket moves. I have sat in that review and nobody looks at where the ticket is actualy going.”Individual Contributor, CXM, Technology engineering and operations

Review capacity was sized for the output of people and received the output of agents. Written responses on review capacity numbered 145, from 135 respondents on 89 engagements, and 134 of the 145 were negative. Respondents described agents that produced more in a morning than two or four reviewers could examine in a day. In those accounts the review queue grew, approval became a formality, or the analysts who produced the output checked it themselves.

“Both sides agreed a checkpoint at the end of each pipeline build, which sounds fine on paper, but in practice it is four of my engineers reading generated code until nine at night. Two asked to move off in month two.”Engagement Manager, ITO, Technology engineering and operations

“Agents produce more test scripts in a morning than we can properly look at, so my day becomes reading output instead of building anything.”Individual Contributor, ITO, Technology engineering and operations

The placement of review points was a separate failure from the depth of review. Written responses on placement numbered 105, from 103 respondents on 75 engagements, and 75 were negative. Some described a review point placed correctly and reduced to a sign-off. Others described review at the end of the process, after an error had already gone out, and output checked solely by the analyst who produced it.

“Here the checks are at the end, after the validation pack is assembled, and by then two of my engineers have spent a day on a result the quant desk was never going to accept. We raise this in the monthly, both sides agree, and the review points stay exactly where they are.”Engagement Manager, ITO, Trade, settlement and reconciliation

“So I spend a fair amount of my time protecting those four from becoming the permanent review desk, while also being the person who told the client we check everything.”Engagement Manager, BPO, Corporate and back-office functions

A review point that approves without examining the output is recorded as a human check in every status report built on it. The control appears to operate in the reporting while it does not operate on the output.

Oversight of agent work requires a view of what the agents did. In the written responses, each firm’s account of agent actions stayed inside its own tools. The tools reported their own success and kept too little trace to locate a failure.

Visibility of agent actions across the two firms was described in 66 written responses, from 64 respondents on 49 engagements, and 62 of the 66 were negative. An account of an agent’s action, when one was needed, was assembled by hand after the event.

“It took eleven days to establish whose change board owned the question, by which time three monthly attribution packs had gone to the client’s investment committee. Neither firm could show the other what had actually been done in that window. We closed it by recalculating the quarter by hand.”Portfolio Director, KPO, Trade, settlement and reconciliation

The tools reported success when a run had failed, produced outputs without a reason attached and logged a start and a finish without the steps between. Written responses on these three failures numbered 73, 163 and 101, and the negative responses numbered 63, 145 and 82 respectively.

Reliability itself was sustained by people. Written responses on the factual reliability of agent output numbered 628, from 14 survey questions, and 548 of the 628 were negative. Respondents described output corrected by delivery staff before it went on and corrections that went unrecorded.

Knowledge of where staff had quietly fixed third-party problems was described in 21 written responses, and 20 were negative. Respondents said the fixes had never been written down and that the people who made them had left.

Oversight in these engagements watched the agents’ reports of their own runs. The runs themselves, and the human corrections within them, appeared in neither firm’s oversight reporting on the respondents’ account.

2.1 Agent Actions Hidden Between the Two Firms

Each firm’s agents acted on shared tasks, and each firm’s account of the agents’ actions stayed inside its own tools. When a question arose about what an agent had done, the analyst nearest the problem reconstructed the answer from logs and statements. Negative responses numbered 62 of the 66 written responses on visibility of agent actions across the two firms.

“They ask me on Tuesday why an alert closed on Sunday and I reconstruct it from logs, and by then the answer is half a guess. Nobody designed for that question.”Engagement Manager, BPO, Financial crime and client onboarding

“When something the agent did needs explaining i cant see there side of it, so i rebuild the break from the custodian statement myself and work back from that. Takes a morning. My team does the same.”Individual Contributor, BPO, Trade, settlement and reconciliation

Changes to shared agents reached the other firm through the output itself. Written responses on agent updates disclosed after the fact numbered 48, from 48 respondents on 44 engagements, and 38 were negative. Respondents described a change made by one firm and found by the other from the output, from a failure or at the next weekly meeting. Some respondents acknowledged that their own firm did the same.

“Someone on their side swapped out the knowledge articles for the AP desk over a weekend and nobody told us, so Monday morning the assist tool was feeding my team the old instructions on invoice holds.”Individual Contributor, CXM, Client servicing and communications

“when their platform team adjusts an agent’s behaviour we generally learn of it afterwards, from a variance in the daily account administration numbers rather than from any notice, and I would not claim our own reporting shows the client anything better in the other direction.”Portfolio Director, BPO, Corporate and back-office functions

Shared visibility existed in a small number of engagements. Positive responses numbered 4 of the 66 on visibility, and they described both firms able to pull the full trace of an agent’s action.

“In nine years of running accounts I have not co-managed one this cleanly – both sides can pull the full trace of what a bot did on a ticket, which removes most arguments before they start.”Engagement Manager, CXM, Corporate and back-office functions

2.2 Tools That Reported Their Own Success

The indicators of observability were distressed in about a third of their answers, against about half for the categories within the reach of the two firms. False completion was distressed in 251 of 786 answers (31.93%), explainability in 262 of 800 (32.75%) and logs and telemetry in 255 of 796 (32.04%). The evidence on the three failures rests chiefly on the written responses.

The tools reported runs, cases and checks as complete when they were not. Written responses on false completion numbered 73, and 63 of the 73 were negative. Every status report and dashboard built on a run inherited its reported success, and the remaining control was a person who repeated the check.

“In April we took a batch of eighty investigations to the bank’s financial crime committee and the tool’s own run summary said all eighty had completed cleanly. Six had not, they had stopped part way and reported success.”Portfolio Director, BPO, Financial crime and client onboarding

“It marks a run complete and the run is not complete, half the sources were never opened. And the logs dont tell you which ones. We only find out because an analyst reads it line by line.”Engagement Manager, KPO, Corporate and back-office functions

The tools also kept too little trace to explain an output or locate a failure. Written responses on explainability numbered 163, and 145 of the 163 were negative. Written responses on logs and telemetry numbered 101, and 82 of the 101 were negative. Respondents described traces that recorded a decision and little else, and logs that showed a start time and an end time and nothing between.

“Here, when the bot resets the wrong entitlement, the trace tells me it made a decision and very little else, so we end up rebuilding the story from ticket history and somebody’s memory.”Engagement Manager, ITO, Technology engineering and operations

“Logs from Platform show start time and end time and nothing between. When it does finish, numbers are mostly right, ill give it that. But i cant prove anything to my lead, i can only re-run it and show him the third one.”Individual Contributor, KPO, Corporate and back-office functions

Tools that reported failure accurately changed the account. Positive responses numbered 10 on false completion and 18 on each of explainability and logs, and they described tools that flagged a failed step without a search through the logs.

“the tools are the strongest thing on this account, they tell us straight when a step has failed and that is worth a lot, we are not digging through logs to find out.”Individual Contributor, ITO, Technology engineering and operations

2.3 Reliability Sustained by Corrections No One Recorded

The reliability of agent output rested on corrections by delivery staff. Written responses on factual reliability numbered 628, from 14 survey questions, and 548 of the 628 were negative. Respondents described output that staff corrected before it went on, corrections that were not logged, and vendor measures that counted the corrected output as the agent’s own.

“And when something the agent produces is wrong, we correct it inside the team and raise it internally rather than waiting to see who is accountable on paper.”Engagement Manager, KPO, Corporate and back-office functions

“What I do now is I keep my own sheet, one tab per conduct Rule, with the alert types that always come out wrong and what the correct tagging should be. My teammates copy from it also. It is not official anywhere, for that reason nobody maintain it except me.”Individual Contributor, CXM, Regulatory, audit, tax and legal

Unstable tool behavior added a second layer of correction. Written responses on the stability of tool behavior numbered 71, and 57 of the 71 were negative. The same input gave different results across runs and weeks without any change on the engagement side, and staff chose or corrected the result by hand before release.

“This one answers a fee question fine on Monday and gets it wrong on Wednesday, same question, nothing changed on our side.”Individual Contributor, CXM, Client servicing and communications

On the respondents’ account, the vendor’s reporting on these engagements described a more reliable agent than the one in operation. The difference was the delivery team’s unrecorded correction, measured as manual alteration of AI outputs (Part 5).

A control without an owner does not operate. On the delivery teams’ account, the engagements lacked an owner at each level where a control needed one, from the financial institution’s ownership of AI to the analyst who handled the agent’s error.

The gaps appeared at four levels in the written responses. Ownership of AI at the financial institution was described in 46 written responses, and 41 were negative. The boundary of agent autonomy was described in 48, and 34 were negative. The authority to stop a misbehaving agent was described in 73, and 59 were negative. Accountability for problems arising from agent action was described in 51, and 45 were negative.

“Procurement operations, their vendor risk function and the technology group each hold a piece, so a question about whether an agent may close a supplier onboarding case without a human goes round all three and comes back a fortnight later, usually unanswered.”Engagement Manager, BPO, Corporate and back-office functions

Decision rights in the financial institution’s management of the engagement held the largest written base among the 10 indicators on authority and accountability. Its 196 written responses came from 173 respondents on 104 engagements, and 184 of the 196 responses (93.88%) were negative. Respondents described decisions that waited because no person at the financial institution owned them.

Where an owner was missing, the decision, the stop and the error each settled on the delivery team. The delivery team decided in the financial institution’s place, cleared the output of an agent that continued to act, and held the error where no manager senior to it accepted responsibility.

3.1 Decisions Without an Owner

Decisions at the financial institution waited because no person owned the task or held the authority to decide. The written responses on decision rights numbered 196, and 184 of the 196 responses (93.88%) were negative. Respondents described questions passed between functions, answers from staff without authority and decision-makers who changed during the engagement.

“In March we flagged a set of advisor calls where the suitability wording looked off, and it took eleven days to find anyone on their side who would own the decision. Compliance said it sat with the front office supervisor, the supervisor said it was a Compliance call, and the cases just sat in the queue the whole time.”Individual Contributor, CXM, Credit, underwriting and suitability

“They answer email fast, Ill give them that, but a fast answer from someone with no authority doesnt move anything. Their lead on this changed twice in eight weeks, so we explained the same design twice to two different people.”Individual Contributor, ITO, Regulatory, audit, tax and legal

The same gap appeared at the handoffs of shared processes. Written responses on workflow ownership numbered 82, from 81 respondents on 58 engagements, and 70 were negative. Where an agent, the vendor’s staff and the financial institution’s staff each performed part of a process, a step that belonged to neither firm became the point where tasks waited and errors went unclaimed.

The delivery team made the decisions that waited. Respondents described individual contributors deciding cases that neither firm had assigned, and decisions taken by the delivery team without a ruling from the financial institution.

“Nobody on either side will say who decides, so it lands on us. I am four years in and I am the one deciding if a income calc is safe to pass. Thats not my call to make and it keeps me up.”Individual Contributor, BPO, Credit, underwriting and suitability

“when merchant raises chargeback dispute and agent picks it up halfway, nobody has written down who owns case after that point. My team decides between ourselves, usually works out. And nobody has ever asked us how we decide it”Individual Contributor, CXM, Payments, cards and disputes

3.2 No Authority to Stop an Agent

The stop of a misbehaving agent depended on named authority. Written responses on the authority to stop an agent numbered 73, from 71 respondents on 51 engagements, and 59 were negative. Where a named person on each side held the authority and could be reached, respondents described an agent stopped in minutes. Without assigned authority, or with authority held by staff working office hours, the agent kept acting while the firms settled who could stop it.

“The agent regenerated our regression pack overnight and it started raising defect ticket on the bank side for things that were not defect at all. Forty something tickets before anyone noticed. To stop it we had to ring my lead, he had to ring their test manager, and by then the morning run was already gone.”Individual Contributor, ITO, Technology engineering and operations

“problem is at night no one over there can authorise stopping the triage bot, i have to raise ticket and wait for morning shift. one named person each side per shift with authority to pause the same, that is all it needs.”Individual Contributor, ITO, Technology engineering and operations

The difference from an error made by a person is pace. An agent’s error continues across records until someone with authority pauses the agent. Written responses on resolution speed against agent error numbered 18, and 16 were negative. Written responses on agent action beyond authorized scope numbered 64, and 48 were negative. Respondents described agents that closed tickets and alerts and changed dozens of records in one night.

“A funding rule change sat with them for four days… And in that time the agent kept releasing on the old rule, so we missed the cutoff twice and i squared it by hand after. Nobody could tell me who was allowed to stop it.”Individual Contributor, BPO, Trade, settlement and reconciliation

The delivery team cleared the agent’s output by hand once the agent stopped. Where the stop itself worked, respondents placed the remaining difficulty in deciding afterward which firm owned the harm.

“We can stop an agent in minutes, that part is solved and both sides know the number to call. What is not solved is who pays when it has already sent two hundred wrong merchant replies before we stopped it.”Engagement Manager, CXM, Payments, cards and disputes

3.3 Accountability Left With the Analyst on Shift

Accountability for an agent’s error passed through each hand the output touched and settled on none. Written responses on accountability for problems arising from agent action numbered 51, from 50 respondents on 37 engagements, and 45 were negative. Respondents described days spent deciding whose fault an error was before either firm fixed it, and incident reports that named a symptom in place of an owner.

“Agent drafts a clause comparison, our reviewer accepts it, client’s counsel signs, and by the time anyone notices the summary dropped a termination right, three people have touched it and each did exactly what was asked of them.”Engagement Manager, KPO, Regulatory, audit, tax and legal

Within the vendor, accountability settled at the lowest level of the delivery team. Written responses on the vendor’s accountability for agent error numbered 38, and 29 were negative. The negative responses described errors that no manager senior to the delivery team would own, held by the analyst on shift.

“And when the agent closed out a real intrusion alert as noise, nobody upstairs put their name on it. That falls on the analyst who was on shift. Thats me some weeks.”Individual Contributor, ITO, Technology engineering and operations

“If an agent gets a hit wrong, the case file carries my name, not the company’s. And nobody above me has said different”Individual Contributor, KPO, Financial crime and client onboarding

The positive responses, 8 of the 38, described a lead or the vendor’s management that accepted the error without passing it down. Respondents who described leadership silence after an error placed the damage in the silence as much as in the error.

“When model produced a summary that missed an indemnity cap and client raised it, our leadership went quiet for the better part of two weeks. That silence did more damage than the error.”Engagement Manager, KPO, Regulatory, audit, tax and legal

“when the agent got a benefits answer wrong last month my employer did not put it on me… they fixed the macro and my lead handled the rest. That counts for something.”Individual Contributor, CXM, Corporate and back-office functions

Change control in these engagements was built for periodic releases and applied to agents that changed several times a day. On the respondents’ account, the delivery team changed first and sought approval afterward. Drift between releases and changes by third-party providers reached the agents without any control detecting them.

“Their change advisory board meets Tuesday and Thursday, and the agents do not know that. A remediation we approve on Monday sits nine days before it is allowed anywhere near live … Nobody on either side is behaving badly here, the calendar simply belongs to an older way of working.”Engagement Manager, ITO, Technology engineering and operations

The financial institution’s change control was described in 85 written responses, from 80 respondents on 64 engagements, and 69 were negative. Joint change control was described in 71, from 70 respondents on 53 engagements, and 57 were negative. The complaint in both was tempo, with change boards meeting weekly, fortnightly or monthly against agents that changed within minutes.

Two further gaps lay outside any change process. Changes the agents made to their own behavior between releases were described in 14 written responses, and 12 were negative. The notice third-party providers gave of their changes was described in 160 written responses, and 108 were negative.

The formal change log and the changes that reached the financial institution’s systems therefore diverged. After a change approved later, a drift no forum ruled on or a provider change found in the output, the change log described less than had changed.

4.1 Changes Made First and Approved Later

Change boards met weekly or monthly, and the agents changed daily. Negative responses numbered 69 of the 85 written responses on the financial institution’s change control. Respondents described changes made without waiting for approval as the consequence of the difference in tempo. Sign-off on joint changes took days or weeks while the agent continued on the old rule.

“i put in a change request. It sit for five week. In that time the tool send out maybe forty letters on the old rule and nobody flag it. When the request finally come back approved, the rule is already changed by somebody else, a different way.”Individual Contributor, CXM, Client servicing and communications

“We raised a form in March for adding one benefit code and it is still open, and meanwhile the Cycle runs anyway.”Engagement Manager, BPO, Corporate and back-office functions

Where sign-off took longer than the change, teams built side channels that kept the engagement moving. Respondents described a group chat in place of the change log, a private list of what the agent had changed and changes agreed outside the joint process. The positive responses, 13 on the financial institution’s change control and 11 on joint change control, described a change process quick enough for the agents.

“the joint review is once a week and by then a bad run is four days old, so we run a private list of what the agent changed and revert back on chat when it looks off. that sheet is the real control, not the meeting”Individual Contributor, ITO, Technology engineering and operations

A change log kept by a side channel is a change log the governance forum does not hold. The changes that reached the financial institution’s systems then exceeded those its change process had approved.

4.2 Drift and Provider Change Without Detection

Agent behavior drifted between formal releases, and the drift had no owner and no forum. The 14 written responses on drift between releases were all from the comment field on joint management, and 12 were negative. The firms found the drift in the output or when a pipeline failed.

“What it cannot show is that agent behavior drifts a little between releases and there is no forum that meets fast enough to rule on whether a drift is a fix or a defect. We end up deciding on the desk and telling both sides afterward.”Engagement Manager, CXM, Client servicing and communications

Third-party changes reached the agents as failures. Written responses on notice of provider change numbered 160, and 108 were negative. Written responses on the routing of notices within the engagement numbered 56, and 42 were negative. Some providers changed what the agents depended on without notice, and others gave notice that stopped at a mailbox or with one person before it reached the people who operated the agents.

“In February the index vendor changes their file layout on a Friday with no notice and our attribution run fails on the Monday morning, we learn it from the output rather than from them.”Engagement Manager, KPO, Trade, settlement and reconciliation

“What has not changed is that the notice goes to some mailbox in our own estate and not to the four people who actually operate the agents, so we hear about it late even when the provider told us in good time”Engagement Manager, CXM, Corporate and back-office functions

A silent failure cost more than a loud one. Written responses on protection against silent breakage numbered 136, and 100 were negative. A failure that stopped the agent cost a morning. A failure the agent ran through reached reconciliations, screening and access control before a person found it, days later.

Engagements with an inexpensive check of their own inputs caught changes in minutes. Written responses on the monitoring of third-party inputs numbered 46, and the 16 positive responses described a morning test case or an alert on the structure of a feed. Respondents described checks built by an individual and not set up as a defined control.

“Every morning before close batch starts, we push three known reconciliation cases through and compare against expected output stored from last week. If provider changed something quietly, the check fails and my team knows before the actual run. It costs us maybe twenty minutes a day.”Engagement Manager, BPO, Trade, settlement and reconciliation

Agents entered the financial institution’s systems through borrowed and shared logins in place of identities of their own. Written responses on the provisioning of system access and agent identities numbered 437, from 348 respondents on 135 engagements, and 403 of the 437 responses (92.22%) were negative. Respondents described access requests that waited weeks or months while the engagement continued by other means.

“We asked in the first week for a service account with read access to the deal room, and four months later my engineers are still borrowing a named person’s credentials, with a manual approval every single time.”Engagement Manager, ITO, Technology engineering and operations

The means by which the engagement continued were manual entry, emailed extracts and logins issued to a person and used by an agent. An agent that acts under a person’s credentials is recorded in the financial institution’s systems as that person. The access log then names a member of staff for each action an agent took.

Access also depended on manual checks that the agents displaced. Written responses on the replacement of quiet human fixes with explicit checks numbered 9, and 7 were negative. The few respondents who wrote described checks on stale accounts that were never written down and stopped when the agent took the step over.

5.1 Agents Operating Through Borrowed and Shared Credentials

Access management at the financial institution treated a credential as a person. Respondents described joiner forms, line managers and leaver dates applied to agents, and service accounts that took weeks or months to issue. The negative written responses on access and identities numbered 403 of 437.

“Every agent we place on the trust administration work needs its own credential on the client’s core, and their access management still treats a credential as a person, with a joiner form, a line manager and a leaver date. We have waited eleven weeks for one service account.”Portfolio Director, BPO, Corporate and back-office functions

The engagement continued under a person’s identity in the meantime. Respondents described agents running under an individual contributor’s own login, under a shared service account and under credentials borrowed from a named member of staff.

“Agent still runs under my own login because nobody decides what identity it gets.”Individual Contributor, ITO, Technology engineering and operations

“Agent identities were requested in March and half were still pending in July, so the agents run under a shared service account, which their own risk people have flagged.”Engagement Manager, BPO, Trade, settlement and reconciliation

The handover of a task from people to an agent removed a control on access. The 9 written responses on the replacement of quiet human fixes are a base of anecdote strength, and 7 were negative. They described manual checks on stale accounts that were not written into a check, and a missed format change after which access remained open for a person who had left.

“one thing not asked, the small manual checks we used to do on stale accounts is not written anywhere, so when agent take over that step nobody know it existed”Individual Contributor, ITO, Technology engineering and operations

The governance model as a whole was inherited from staffed delivery and applied unchanged to agents. On the respondents’ account, authority rested with managers senior to the people who met each day. Management processes counted people and hours, and each engagement built its own route to governed production while the service was live.

“Issues do get worked as one team, that part functions. The rest is built for people doing the work. There is no joint record of what the agents changed between reviews, so when a forecast number moves we cannot say which side moved it.”Engagement Manager, KPO, Corporate and back-office functions

The joint governance model was described in 10 written responses, a small base, and 9 were negative. They described a model drawn up for people who performed the tasks by hand and left in place after agents took over a large part of the tasks. The financial institution’s capacity to oversee agent risk was described in 45 written responses, from 44 respondents on 38 engagements, and 37 were negative.

The vendor’s side held the same inheritance. The vendor’s internal management processes were described in 25 written responses, and 23 were negative. Respondents described rosters, headcount reports, utilization targets and checklists written for manual delivery, applied to tasks that agents performed.

“the company runs this like a call center with headcount and shift plans, but there is no headcount now, there is agent making answers and nobody redesigned any of that.”Individual Contributor, CXM, Client servicing and communications

The maturity of the vendor’s playbook for taking agentic use cases to governed production held the largest written base of the four, 200 responses, and 186 of the 200 were negative. Respondents described the route from pilot to governed production as one each engagement built for itself, after the sale and while the service was live.

A governance model built for people assumes that a person performed each task and can be asked afterward why. The agents in these engagements acted at a volume and pace the model was not designed to examine.

6.1 Governance Inherited From Staffed Delivery

The financial institution’s mechanisms for overseeing agent risk were missing or late. The 45 written responses on its capacity to oversee agent risk came mostly from the question on the challenges that reduced job satisfaction. The mechanisms named were model-risk review, audit frameworks and vendor-risk terms, each described as late, absent or written for vendors that staffed their engagements with people. The 7 positive responses described a risk function or an AI oversight forum in place before the engagement began.

“No audit framework existed for a system this hard to inspect”Individual Contributor, ITO, Financial crime and client onboarding

“Vendor-risk framework was never rebuilt for an agentic vendor”Individual Contributor, KPO, Regulatory, audit, tax and legal

The vendor’s own controls were written for manual delivery. Respondents described quality audits that applied release checklists written for people to pipelines with steps generated by agents.

“Last quarter we had internal QA audit on our deploy pipeline. The checklist they used was written for manual releases, so half of it did not apply and there was nothing on it for agent generated steps.”Individual Contributor, ITO, Technology engineering and operations

Each engagement built its own route to governed production. The negative written responses on the vendor’s playbook numbered 186 of 200. Respondents described quality assurance for agent output built after the sale, a playbook written for staffed delivery and a method improvised on live work.

“we are building quality assurance for agent output after we have already sold the work, engagement by engagement, and each team invents its own answer. I have now seen that pattern on four accounts.”Portfolio Director, CXM, Corporate and back-office functions

“When the routing broke on the second week nobody in my company had done this before, consequently we called a person from another account who also didnt know. There is no playbook, we invented one on a whiteboard.”Individual Contributor, CXM, Corporate and back-office functions

6.2 What the Bypassed Controls Leave Out of the Financial Institution’s Records

The controls described in the written responses formed a chain, and the written responses describe a failure at each link. Controls existed in name only, oversight lacked visibility, authority was unassigned, change moved outside control and access ran through borrowed logins. A governance model built for people was applied to all five.

On the respondents’ account, the financial institution’s records of agent work were incomplete at each link. A review point recorded a human check that had not examined the output. A log recorded a start and a finish. A change log omitted changes made through side channels. An access log named a member of staff for an action an agent took. A vendor measure credited the agent with output that a person had corrected.

“The old team had two people who knew the HR system drops the middle name field and you correct it before it goes anywhere, and when the agent took that work over nobody turned it into a written check, it simply stopped happening. We find those when something fails, not before.”Engagement Manager, CXM, Corporate and back-office functions

The gap in these records is where the delivery team’s manual corrections took place. A correction made by hand to data, a process or agent output is invisible to a review point that does not examine output and to a log that records a start and a finish. The extent of that manual alteration, its reasons and its disclosure are measured from the delivery team’s own answers (Part 5).