Recommended Controls

The study reports gaps in financial institution readiness, which covers the data, processes, people, and internal controls needed for delivery. Weaknesses in co-management, the two firms’ arrangements for decisions, risks, changes, and human review, were accompanied by limited visibility into work performed by delivery teams (Parts 2–5). These findings call for controls that establish preparation before commitment and verify how work is actually completed during delivery.

For new engagements, financial institution readiness should be established before contract approval. Co-management should be established before contract signature. For existing engagements, the same controls provide the basis for identifying gaps and prioritizing corrections. The recommendations and schedule are proposed responses to the findings. Their effectiveness was not tested by the study.

1 Establish a Readiness Standard Before Contract Approval

Financial institution readiness received the lowest category rating in the sample. This category covers the financial institution’s data, processes, systems, people, internal controls, and responsiveness. Respondents described difficulties with those conditions, including staffing, decision-making, access, and business-case preparation (Part 2).

Before contract approval, assess data ownership and validation, technical access, documented processes, the identities and access permissions assigned to agents, decision rights, and the business case. Record material gaps, assign owners, and identify those that must be resolved before approval. Specify any remaining conditions that must be satisfied before deployment.

The proposed readiness review is intended to identify work that might otherwise transfer informally to the delivery team. Its effectiveness should be evaluated through implementation and subsequent operational evidence.

Data is bad, this is true, but is not fault of people on their side. Systems are old and nobody there build them, they inherit. When we ask for someone who know the data, they find person same day usually, i cannot complain about that. Access also came through quicker than i expect. What they will not do is change process, they want agent to do exact same steps as before, and for governing AI risk nobody there has done this before, so of course is not mature yet. Two months is not long time.”

INDIVIDUAL CONTRIBUTOR · ITO, Technology engineering and operations

2 Establish Co-Management Before Contract Signature

Accounts of co-management identified separate risk lists, unresolved decisions, and unclear authority to suspend agents (Parts 2 and 3). Positive accounts described arrangements that resolved issues and supported timely intervention.

Before contract signature, agree joint decision rights, limits on the actions agents may take, escalation routes, and shared risk and issue records. Confirm that these arrangements operate before delivery begins. Assign reachable personnel with authority to suspend operation whenever agents are active, with documented coverage and escalation if those personnel are unavailable.

These arrangements should be tested in realistic scenarios. The study provides reasons to examine them but does not establish their implementation cost or guarantee a particular operational benefit.

Step ownership is much clearer now than at kickoff. We sat with their servicing lead in month one and wrote down who owns which part of the flow, voice and chat separately, and that document has held. Change visibility also improved, when the assistant's behaviour is adjusted between releases both sides now see it before it goes live, which was not so in first weeks. What has not moved is the stopping decision. If a reply is going out wrong on chat there is no single agreed person who can pull it from the queue, so i have to find their duty manager on call, and that is twenty minutes we do not have. Fault is the same story, after an incident the two firms are still arguing whose it was. Our joint sampling is also too thin to catch much before cardholder sees it.”

ENGAGEMENT MANAGER · CXM, Payments, cards and disputes

3 Audit the Design and Calculation of Vendor Reporting

Respondents described reporting based on activity counts, exclusions, or outputs corrected by delivery teams (Part 5). Such choices can materially affect interpretation even when the underlying arithmetic is correct.

Review reporting definitions, denominators, exclusions, baselines, case aging (the elapsed time since a case was opened), and the treatment of work corrected by people. Reconcile reported results to sampled operational records and check the calculations. Verify agent-generated completion reports against evidence of execution.

Performance measures should distinguish unassisted agent output from work requiring review, correction, or manual completion, with definitions that allow comparison over time.

The vendor's attribution dashboard reports a figure as verified once it passes an internal reconciliation check the vendor itself designs and controls, with no independent audit trail the client can inspect.”

ENGAGEMENT MANAGER · KPO, Trade, settlement and reconciliation

4 Restructure Commercial Terms Around Outcomes and Exit

Accounts of pricing based on delivery team headcount, uncertain licensing, and limited tested alternatives suggest areas for commercial review (Part 2). The study did not establish that any one pricing model caused concealment or delivery failure.

Align commercial terms with measurable service outcomes and disclose the human effort included in delivery. Confirm permissions for agent use of third-party services and document feasible transition or exit arrangements. Test those arrangements to an extent proportionate to the consequences of losing the vendor or a relevant model provider or third-party provider.

Outcome-based terms may improve alignment, but they can also create incentives to select favorable measures. Reporting definitions, verification rights, and transparency about manual effort remain necessary under any pricing structure.

When we discussed about the second cycle estimate in March, it was still built on how many analysts sit on the work, which does not fit when the model does the first pass, and I marked those items low for that reason. To be fair, when a calibration run went wrong my leadership did not push it down to my team, they took it up themselves. The banking training is thin though. Most of what my modellers know about the client's chart of accounts they learned from the client.”

ENGAGEMENT MANAGER · KPO, Corporate and back-office functions

5 Treat Human Oversight as a Control Function

Respondents described inadequate review capacity, superficial approval, and change procedures poorly matched to operational needs (Parts 2 and 3). These accounts support testing how oversight functions in practice.

Design, staff, and test human oversight according to output volume, risk, complexity, and the time required for meaningful review. Record what reviewers examine. Independently repeat the review work for a risk-based sample of approvals, a procedure known as reperformance, to assess the quality of scrutiny.

Use risk-based change procedures with defined approval, notification, emergency handling, and subsequent reconciliation of emergency changes with the approved records. Assign responsibility for detecting unexpected agent behavior and investigating changes in inputs, configurations, or services supplied by the vendor, model provider, or third-party providers.

The suspicious-activity dashboard reports case-closure volume against target without separately reporting how many closures were auto-approved agent recommendations versus human-reviewed decisions, which hides the real oversight bottleneck from anyone reading the summary.”

INDIVIDUAL CONTRIBUTOR · BPO, Financial crime and client onboarding

6 Listen Directly to the Vendor Delivery Team

Evidence obtained directly from the delivery team should complement management reporting because the roles reported different aspects of manual alteration and its discussion. Individual contributors supplied most affirmative answers about personally performing manual alteration. Engagement managers supplied all 13 affirmative answers about discussion with the financial institution across the three structured questions on data, processes, and AI outputs (Parts 4 and 5). These are pooled answer counts, not counts of unique people or engagements. The pattern does not establish individual contributors' overall contact with the financial institution.

Establish appropriate channels for delivery team members to describe operational problems, manual alteration, and suggestions. Options include confidential interviews, periodic surveys, and review of delivery team records. Define access, privacy, protection against retaliation, escalation, and the limits of confidentiality before collecting information.

Accounts from the delivery team may identify issues not apparent in aggregate reports. They should be assessed alongside operational evidence and information from other relevant parties.

We are sitting till midnight most cycles because their side answer nothing. I miss two family functions for a deadline that move anyway… my team is tired and two of the juniors are already applying somewhere else. It is not only the work that suffer”

INDIVIDUAL CONTRIBUTOR · KPO, Corporate and back-office functions

7 Require a Manual Alteration Register and Test It

At least one form of manual alteration was reported on every sampled engagement in the combined direct answers and written accounts reported in Part 5. This does not mean that every method occurred on every engagement. Accounts of incomplete recording and limited discussion support reviewing how manual alteration is authorized, documented, and reported.

Establish a manual alteration register, a shared record of material manual alteration and its authorization. Cover manual alteration of financial-institution-provided data, manual alteration of financial institution processes, manual alteration of AI outputs, reporting alteration, and third-party work done by hand. Define materiality and record routine low-risk corrections through proportionate workflow logging linked to the register. Together, the records should identify what was done, by whom, why, under what authority, to which output, and with what required communication.

Test the completeness of these records by tracing samples of outputs, data loads, and reported results to documented manual alteration. Reconcile local trackers with shared records where appropriate and investigate unexplained differences.

This is close to constant on this engagement at this point. We reconcile document versions and fix formatting nearly every week, we have told our own leadership and we have told the client in writing, and nothing has really changed on their end. It has become the baseline expectation rather than an exception at this point.”

ENGAGEMENT MANAGER · KPO, Corporate and back-office functions · “

The vendor's change log recorded release approvals as complete once submitted for sign-off, not once actually approved, so the client-facing change-control dashboard showed a healthy pace of updates even during weeks when several releases sat pending for days.”

INDIVIDUAL CONTRIBUTOR

BPO, Corporate and back-office functions