Manual Alteration by the Delivery Team

The delivery team sustained the engagements by altering their inputs, processes and outputs by hand. The study measures five manual alteration methods. They cover the data the financial institution provided, its processes and the outputs of the AI tools, together with the reporting it received and third-party work done by hand. Across all five methods, manual alteration was described on 150 of the 150 engagements. On the three methods counted by a direct question, 619 respondents on 146 engagements reported at least one.

“i do this almost every day now. i inform my team lead please, not the client, therefore it stays internal. it is exhausting but i see no other way.”Individual Contributor, CXM, Corporate and back-office functions

Manual alteration was routine work, performed by individual contributors in response to the failures reported in the earlier parts of the study. It was discussed with vendor management in 783 of 1,113 answers (70.35%) and with the financial institution in 13 of 1,004. In the written answers it was seldom recorded.

“We do not report every fix as a ticket, only bigger ones, besides manager already know about most of these informations, so nothing is really hidden, it is just quiet work every week”Individual Contributor, ITO, Technology engineering and operations

Two limits bind this analysis. The study records what respondents did and whom they told, and it never records what the financial institution knew. The study cannot establish that any manual alteration breached a legal or regulatory obligation, and the regulatory and legal obligations it reports are context for an audit. The evidence comes from vendor delivery staff alone, and no financial institution was approached or asked for comment.

Manual alteration has five objects. The delivery team altered the data the financial institution provided, the financial institution’s processes and the outputs of the AI tools. It altered the reporting the financial institution received, and it performed by hand the tasks that third-party systems did not deliver. Three of the five methods were counted by a direct question to each respondent.

The three counted methods reached most engagements. Manual alteration of financial institution processes was reported by 490 of 841 respondents (58.26%) on 132 engagements, and of financial-institution-provided data by 421 of 839 (50.18%) on 127 engagements. Manual alteration of AI outputs was reported by 278 of 828 respondents (33.57%) on 108 engagements.

Engagements where manual alteration was reported most were worse on every study measure. For each counted method, engagements assigned the answer True had a higher cancellation rate, lower customer satisfaction among active engagements and a lower engagement indicator mean. Their respondents reported higher intent to leave. The written answers describe manual alteration as a response to the same failures and not as their origin.

Each engagement is assigned the answer given by most of its respondents. Engagements with a tied answer have no assigned answer and are not in the table.

Each counted method has a companion question on the wider team. Respondents rated three statements that their greater team “regularly alters” the data, the processes and the AI outputs, on a scale with no good or bad end. Stronger agreement on each was associated with more cancellation, with correlations from r = 0.59 to r = 0.70 (n = 150 engagements).

1.1 Financial-Institution-Provided Data

Half of the respondents who answered had altered the data the financial institution provided, 421 of the 839 respondents who answered (50.18%). The wider team’s regular alteration of the data had a mean rating of 5.4 (n = 773 respondents) and was associated with the engagement indicator mean at r = −0.84 (n = 150 engagements). The two questions describe the same practice from the respondent and from the team.

The manual alteration answers the data failures reported with financial institution readiness (Part 2). Data arrived with missing fields and duplicate entries every cycle, and the delivery team lacked the access to correct it at its source. The team corrected it by hand before the agents could use it, and the next delivery arrived with the same faults.

“I fix wrong Data every day, team lead know but we do not raise it with client, feedbacks would go nowhere anyway”Individual Contributor, BPO, Trade, settlement and reconciliation

“I flag it every week, my manager knows but nothing changes upstream. This Vendor takes zero ownership so we are the ones patching data every single day”Individual Contributor, CXM, Technology engineering and operations

“i correct the extracts before loading them, half the fields come swapped or blank before we even open the file,”Individual Contributor, ITO, Technology engineering and operations

1.1.1 Manual Alteration of Financial-Institution-Provided Data

Respondents answered the statement “I have altered client-provided data.” True was the answer of 421 of the 839 respondents who answered (50.18%), and False of 418 (49.82%). The share answering True was 69.62% on canceled engagements (n = 316 respondents) and 36.59% on active engagements in Transition/Cut-Over (n = 246 respondents).

At engagement level, the answer was associated with all four study measures. Cancellation reached 61.84% of the 76 engagements assigned the answer True, against 10.17% of the 59 engagements assigned False. Every association held under each of the robustness checks.

The 578 written responses came from 378 respondents on 128 engagements, and 571 were negative. Respondents who explained why they altered the data described data that arrived unusable every cycle, after earlier cleaning, and access too limited to correct it at the source.

“Last cycle their file was missing tax ids for half the batch, I fixed it myself before run started… nothing gets checked before it lands on our desk, this happens basically every cycle now”Individual Contributor, BPO, Corporate and back-office functions

“Last cycle it was three whole cohort missing from one file, I only catch it because I count row against last cycle total and numbers didnt match up, if I hadnt counted it would have gone straight through with people missing pay.”Individual Contributor, BPO, Corporate and back-office functions

“last month the cash statement came after cut off again… so i take previous day file and roll forward, thus the break clears on time.”Individual Contributor, BPO, Trade, settlement and reconciliation

1.1.2 Regular Alteration of Financial-Institution-Provided Data by the Wider Team

Respondents rated the statement “My greater team regularly alters client provided data” on a scale with no good or bad end. The mean rating was 5.4 and the median 5.0 (n = 773 respondents). Respondents on canceled engagements gave a mean of 6.9 (n = 287 respondents).

Stronger agreement was associated with more cancellation (r = 0.70, n = 150 engagements) and a lower engagement indicator mean (r = −0.84, n = 150 engagements). It was also associated with lower customer satisfaction among active engagements (r = −0.65, n = 94 engagements) and higher intent to leave (r = 0.66, n = 722 respondents). None of the four associations failed a robustness check.

The 84 written responses included 83 negative. They described data cleaned every cycle that arrived in error again the next, alteration known to the team lead, seldom logged and seldom reported to the financial institution. The single positive response described a data exception step agreed with the financial institution’s platform owner before any correction was applied.

“I keep note of fixes i make each sprint, my lead know about it, client does not. Take maybe extra hour every day and i do not think should be my job to clean their data before agent see it.”Individual Contributor, ITO, Technology engineering and operations

1.2 Financial Institution Processes

Manual alteration of processes was the most reported of the three counted methods, by 490 of the 841 respondents who answered (58.26%). It was the delivery team’s answer to decisions that did not come and to approval cycles slower than the tasks they governed. The wider team’s regular alteration of processes had a mean rating of 5.7 (n = 779 respondents).

The alteration follows from the failures of control reported earlier. Joint change control met weekly or monthly against agents that changed daily, and decisions at the financial institution waited for an owner (Part 2). The delivery team changed the process first and sought approval afterward, and the change log described less than had changed (Part 3).

“nobody upstream will say yes or no so i just go with my own call, decision take too long to come back and Payroll deadline is same every cycle so i cannot wait”Individual Contributor, BPO, Corporate and back-office functions

“This is basically every week now, my manager knows about it. Ticket takes days to clear, fix itself takes an hour, so waiting on the ticket process makes no sense when the job needs doing”Individual Contributor, ITO, Technology engineering and operations

1.2.1 Manual Alteration of Financial Institution Processes

Respondents answered the statement “I have altered client processes.” True was the answer of 490 of the 841 respondents who answered (58.26%), and False of 351 (41.74%). The share answering True was 73.73% on canceled engagements (n = 316 respondents) and 45.14% on active engagements in Build-Out (n = 175 respondents). It also differed by outsourcing model, from 69.42% among respondents on BPO engagements (n = 206) to 50.87% on CXM engagements (n = 230).

The answer assigned to each engagement was associated with each of the four study measures. Cancellation reached 50.00% of the 94 engagements assigned the answer True, against 15.22% of the 46 assigned False. The associations survived every robustness check applied.

The 849 written responses came from 513 respondents on 137 engagements, and 841 were negative. Respondents described decisions that no person at the financial institution would make in time, change approval built for one change a month, and procedures never written down or not matching practice. The change was made first and recorded later or not at all.

“Escalating a break means waiting on someone who never actually rules on it, we just pick a call and run with it. And whatever that forces into the workflow gets written up later, if it gets written up at all.”Individual Contributor, BPO, Trade, settlement and reconciliation

“i changed a step on the reconciliation process last week, nothing about the change process here really matches how the agent behaves day to day, so we adjust as we go and i discuss about it with my lead, its not a big drama”Individual Contributor, BPO, Trade, settlement and reconciliation

1.2.2 Regular Alteration of Financial Institution Processes by the Wider Team

The statement “My greater team regularly alters client processes” was rated on a scale with no good or bad end. The mean rating was 5.7 and the median 6.0 (n = 779 respondents). Respondents on canceled engagements gave a mean of 7.1 (n = 291 respondents).

Stronger agreement was associated with a lower engagement indicator mean (r = −0.83, n = 150 engagements) and more cancellation (r = 0.63, n = 150 engagements). It was also associated with lower customer satisfaction among active engagements (r = −0.64, n = 94 engagements) and higher intent to leave (r = 0.65, n = 730 respondents).

The 111 written responses included 110 negative. Respondents described process change as routine, decisions that did not come made by the team, and the change known to the team lead and seldom reported to the financial institution.

“This is normal now, my team decide small things ourselves and tell manager after not before. I don’t ask if client know, not my place. Happens most weeks, only way to hit the cycle.”Individual Contributor, BPO, Corporate and back-office functions

1.3 AI Outputs

The correction of AI outputs was the least reported of the three counted methods, by 278 of the 828 respondents who answered (33.57%). It was also the method most closely tied to cancellation. Cancellation reached 76.19% of the 42 engagements assigned the answer True.

The correction kept delivered output clean and left no trace of the effort behind it. The same input gave different results from one run to the next, and the person handling the case chose the version that looked right and corrected it before it went on. The vendor’s measures counted the corrected output as the agent’s own (Part 3).

“Nobody trusts the raw output so I rewrite it every time, that’s the real Job now not what they think we do”Individual Contributor, CXM, Corporate and back-office functions

“the problem is nobody talk about how often team do this because everyone just fix quietly on their own, do understand I only track for myself, I do it almost every case I recieve”Individual Contributor, BPO, Corporate and back-office functions

1.3.1 Manual Alteration of AI Outputs

Respondents answered the statement “I have altered AI outputs.” True was the answer of 278 of the 828 respondents who answered (33.57%), and False of 550 (66.43%). The share answering True was 54.19% on canceled engagements (n = 310 respondents) and 18.52% on active engagements in Transition/Cut-Over (n = 243 respondents).

Engagement-level tests found the answer associated with every study measure. Cancellation reached 76.19% of the 42 engagements assigned the answer True, against 22.00% of the 100 assigned False. No robustness check overturned an association.

The 407 written responses came from 276 respondents on 116 engagements, and 399 were negative. Respondents described an agent that gave a different answer to the same input from one run to the next. They chose a version by hand and corrected it before it moved on, ahead of an approval slower than the deadline.

“the problem is the ai gives a different match each time i run it, so i correct it myself before it moves on, on that basis i do not wait for approval”Individual Contributor, BPO, Trade, settlement and reconciliation

“So what I do is run it, then run it again, and if the two agree I go with it, and if they don’t I work it out by hand. That’s my process. Nobody designed that, I invented it because I needed something.”Individual Contributor, BPO, Trade, settlement and reconciliation

“the problem is change control takes weeks for anything, so whats the point raising it every time, i patch the output myself and keep the queue moving, thats just how this job works and truthfully im quicker at catching it than the process is anyway”Individual Contributor, CXM, Corporate and back-office functions

1.3.2 Regular Alteration of AI Outputs by the Wider Team

On a scale with no good or bad end, respondents rated the statement “My greater team regularly alters AI outputs.” The mean rating was 4.4 and the median 4.0 (n = 665 respondents), the lowest agreement of the three wider-team statements. Respondents on canceled engagements gave a mean of 5.8 (n = 228 respondents).

Agreement tracked more cancellation (r = 0.59, n = 150 engagements) and higher intent to leave (r = 0.58, n = 620 respondents). Its association with customer satisfaction among active engagements, r = −0.40 (n = 94 engagements), was the weakest of the three wider-team statements.

All 23 written responses were negative. The 19 that commented on the alteration of AI outputs described correction as constant team effort, done before anyone outside the team saw the output and seldom logged.

“it’s f***ing unpredictable, same input different answer. i fix it quiet every time, my lead knows, client doesn’t. that’s just how it is here.”Individual Contributor, CXM, Corporate and back-office functions

1.4 Reporting to the Financial Institution

Reporting alteration is the fourth method, the way the reports and dashboards the financial institution received were built. Asked how the vendor alters the performance reports and dashboards given to the financial institution, 484 of 775 written answers (62.45%) gave an answer other than “I do not know,” on 127 engagements. The remaining 291 answers, on 95 engagements, were “I do not know.” The reports, on the respondents’ account, showed the agent’s activity in place of its accuracy.

“When the number came back at nine percent nobody above me asked what happened on the desk, they asked how it would be presented. Therefore my people learned that the honest answer is not the wanted answer, and two of them are already looking.”Engagement Manager, CXM, Corporate and back-office functions

The written answers name the mechanisms. Among the 475 substantive answers on the respondent’s own engagement, an activity count reported in place of an outcome was described in 109. Human-corrected work counted as the agent’s was described in 51, and failures removed from the base of a reported rate in 49. Measures restricted to the easy cases appeared in 38, exceptions recorded internally and withheld from the financial institution in 23, and open cases pooled so they never aged in 3.

“The vendor rolled settlement fails that were manually corrected by a human into the same success bucket as fails the agent resolved on its own, which overstated how much the agent was actually doing correctly.”Individual Contributor, BPO, Trade, settlement and reconciliation

“The vendor’s settlement dashboard reported the fails rate as a percentage of trades processed that day rather than trades still open, so a growing backlog of unresolved fails never showed up as a rising number.”Engagement Manager, BPO, Trade, settlement and reconciliation

“The help-desk dashboard reports agent deflection rate as tickets the agent handled without escalation, but tickets the agent closes and the same employee reopens within 24 hours are counted as a new, separate ticket rather than a reopened one, which hides the actual repeat-contact rate the deflection number is supposed to reflect.”Engagement Manager, CXM, Corporate and back-office functions

Reporting alteration connects the other four methods to the financial institution’s view of the engagement. A correction made by hand to data, a process or an output, and work done by hand for a third party, reached the financial institution through reports that counted the result as the agent’s. The ratings on reporting are reported as ratings and never as established fact.

1.4.1 Accuracy of Dashboards and Performance Reports

Respondents rated the statement “The dashboard and performance reports given to the customer are accurate.” The mean rating was 4.8 and the median 5.0 (n = 794 respondents), and 220 of the 794 answers (27.71%) were distressed. The 301 respondents on canceled engagements rated the reports’ accuracy at a mean of 3.5.

Respondents who rated the reports as more accurate were on engagements with less cancellation (r = −0.61, n = 150 engagements) and a higher engagement indicator mean (r = 0.81, n = 150 engagements). All four associations held under the 22 robustness checks.

The 468 written responses came from 453 respondents on 125 engagements, and 467 were negative. Respondents described measures that counted a task as done when the agent acted, omitted the cases a person had to correct, and blended or reclassified failures until the headline figure looked sound. The single positive response valued figures the respondent could now defend.

“The vendor’s status dashboard counted a task as complete the moment the agent submitted an action, before any human check confirmed the pay run was correct, so failed items looked closed until someone dug into the underlying case log.”Individual Contributor, BPO, Corporate and back-office functions

“The vendor’s weekly report calculates average handle time using only tickets the agent fully resolves on its own, so the growing share of tickets the agent partially processes and then silently hands back to a human queue never enters the handle-time calculation at all, keeping the headline average artificially low.”Individual Contributor, CXM, Corporate and back-office functions

“The vendor’s monthly billing accuracy dashboard reports fee exceptions only after they clear a manual override queue, so exceptions the agent catches and auto-corrects same-day never post as exceptions at all, making the vendor’s own error rate look better than the fee leakage the agent is actually finding.”Engagement Manager, BPO, Payments, cards and disputes

1.4.2 Vendor Alteration of Dashboards and Performance Reports

Respondents rated the statement “The vendor manipulates the dashboard and performance reports given to the customer.” On the goodness view, on which a higher figure means less alteration, the mean rating was 4.7 and the median 5.0 (n = 786 respondents). The distressed range held 235 of the 786 answers (29.90%), and respondents on canceled engagements gave a mean of 3.4 on the goodness view (n = 302 respondents).

Ratings nearer the good end, reporting less alteration, were associated with less cancellation (r = −0.61, n = 150 engagements) and a higher engagement indicator mean (r = 0.79, n = 150 engagements). The rating is the respondents’ judgment of their employer’s reporting and is reported as a rating.

The 465 written responses came from 456 respondents on 125 engagements, and 464 were negative. They described reporting built so that the reported figure held, by the mechanisms of the activity count, the exclusion of failures and the counting of human-corrected work as the agent’s.

“Purchase requisitions the agent could not match to a valid supplier record were logged as processed with a manual override flag that never surfaced in the weekly summary the customer received.”Individual Contributor, CXM, Corporate and back-office functions

“The vendor’s exception dashboard groups unresolved custody breaks under a catch-all pending category that does not age, so a break sitting open for weeks looks the same as one opened yesterday.”Individual Contributor, BPO, Trade, settlement and reconciliation

“The vendor’s monthly performance report calculates first-contact resolution using only tickets the agent both received and closed within the same reporting window, so any ticket the agent picks up late in the month and carries into the next cycle simply disappears from both months’ denominators instead of counting as a miss.”Individual Contributor, CXM, Technology engineering and operations

“The vendor’s weekly ops dashboard tracks fee break resolution time starting from ticket creation rather than from when the agent first flagged the anomaly internally, so breaks the agent catches and holds for days before a ticket gets opened never count against the resolution clock at all.”Individual Contributor, BPO, Payments, cards and disputes

1.4.3 Prevalence of Reporting Alteration Against Five Years Earlier

Respondents rated whether altering the dashboards and performance reports given to the financial institution is more or less common than five years earlier. On the goodness view, the mean rating was 4.8 and the median 5.0 (n = 795 respondents), and 247 of the 795 answers (31.07%) were distressed. Higher ratings on the goodness view were associated with a higher engagement indicator mean (r = 0.79, n = 150 engagements) and less cancellation (r = −0.60, n = 150 engagements).

No respondent wrote about whether the practice had grown more or less common.

1.5 Third-Party Work Done by Hand

Third-party work done by hand completes the five methods. The delivery team performed by hand the tasks a third-party system did not deliver. Respondents described records pulled from a provider portal with no interface, data keyed in where a broken feed left gaps and cases reopened after a provider change. The written answers describe it in 91 answers from 87 respondents on 63 engagements.

“administrator portal is same as day one… still no proper interface, we pull screens by hand at cut off.”Individual Contributor, BPO, Trade, settlement and reconciliation

“Everything we use sits on one account with one provider. When that access got suspended in June nobody has a second way in, my team did the tickets by hand for two days”Individual Contributor, CXM, Corporate and back-office functions

The method is the manual fallback reported with the third-party ecosystem (Part 2). The knowledge of how to perform the tasks by hand, and of where staff had quietly fixed third-party problems, was held in people. Written responses on those quiet fixes numbered 21, and 20 were negative.

“I score that section down for one reason only, nobody wrote down what old ops team use to patch by hand before the monitoring agent came”Individual Contributor, ITO, Technology engineering and operations

Manual alteration was reported almost entirely by individual contributors, the role that handles the data, the process and the output. The highest share among engagement managers was on processes, where 9 of the 148 engagement managers who answered said True (6.08%). No portfolio director answered True to any of the three.

The pattern has two interpretations that the data cannot separate. Alteration may be done at the level where the data, process or output is handled, out of managers’ sight. Managers may also be reluctant to report alteration made under their oversight. Both interpretations leave the alteration with the individual contributors and outside the view of the management line.

Disclosure followed the same division. The few alterations engagement managers reported were discussed with the financial institution in 13 of 14 answers. The alterations individual contributors reported were discussed with the financial institution in none of 990 answers.

2.1 Manual Alteration by Role

The three counted questions were compared across individual contributors, engagement managers and portfolio directors, on the respondents who answered True or False. The two customer management executives gave no True or False answer to any of the three and are not in the comparison.

2.1.1 Manual Alteration Confined Almost Entirely to Individual Contributors

Saying True was almost entirely confined to individual contributors. On financial-institution-provided data, 63.43% of individual contributors answered True (n = 659), against 2.00% of engagement managers (n = 150) and 0.00% of portfolio directors (n = 30). The role difference gave chi-square 215.77 on two degrees of freedom (p < 0.001, n = 839 respondents).

Each test gave p < 0.001. The tests treat respondents as independent, and no clustered test was computed. The p values are therefore approximate.

The analysis cannot separate the two interpretations of the pattern. Work done at the level where it is handled and reluctance among managers to report alteration under their oversight produce the same answers.

2.2 The Account of Engagement Managers and Portfolio Directors

Engagement managers gave 14 True answers across the three counted questions, against 1,175 from individual contributors. Portfolio directors and customer management executives gave none. The few manager alterations sit beside the individual contributors’ alterations with a different record of disclosure.

2.2.1 Manager and Director Alterations Disclosed to the Financial Institution

The two roles differed on disclosure to the financial institution. Engagement managers answered True on discussion with internal management in 14 of 14 answers, and on discussion with the financial institution in 13 of 14. Individual contributors answered True on discussion with internal management in 769 of 1,099 answers (69.97%), and on discussion with the financial institution in none of 990.

A respondent can contribute up to three answers in each group. The counts are therefore answers and are not independent of one another. The engagement managers’ 14 answers are too few for a rate or a test.

“We do flag it, both internally and to the client, but flagging does not stop us doing it, strictly speaking the account still needs the composites finished on time.”Engagement Manager, KPO, Trade, settlement and reconciliation

“we have told the client directly that we alter their data before it reaches the model, and told our own management as well, also I am careful how that gets phrased since altering anything on their side sits close to a line we are not meant to cross without sign-off.”Engagement Manager, CXM, Financial crime and client onboarding

2.3 Manual Alteration Across the Indicator Quintiles

Reported manual alteration followed the engagement indicator mean in five steps. Each respondent takes the indicator quintile of the engagement, from the 30 engagements with the lowest engagement indicator mean in quintile 1 to the 30 with the highest in quintile 5. On each counted method, the share answering True was highest in quintile 1 and lowest in quintile 5.

Shares are of respondents who answered True or False. The tests of the difference across quintiles are at respondent level and not adjusted for clustering within engagements.

The difference across quintiles gave chi-square 131.67 for data, 122.81 for processes and 135.80 for AI outputs (4 df, p < 0.001 for each). For data and AI outputs the share was lower at every step. The gradient for processes had one reversal, between the second and third quintiles. The quintiles are built on the same indicators as the engagement indicator mean (Part 1).

Respondents who answered True to a counted question were asked why they had altered the data, the processes or the AI outputs. The three questions received 418 answers on data, 489 on processes and 276 on AI outputs, each from as many respondents. The reasons named directly the failures reported in the earlier parts of the study.

“Its annoying, Security never signs off my access so the Agent can’t see live PO status, and two Vendor records for the same supplier is normal here not exceptional. So I work off an export thats a day old and I reconcile the duplicates by hand before anything runs. Every morning. Its about forty minutes.”Individual Contributor, CXM, Corporate and back-office functions

The reasons divide by method. In the respondents’ answers, data was altered because it arrived wrong and could not be corrected at its source. Processes were altered because decisions did not come and procedures were never written down. AI outputs were altered because the same input gave different results from one run to the next.

Each reason points to a failure upstream of the delivery team. The alteration is the delivery team’s response to that failure, made at the point where it reached the agents.

3.1 Reasons for Altering Financial-Institution-Provided Data

The reasons for altering data were led by the quality of the data itself. The quality of the financial institution’s data was named in 229 of the 400 substantive answers on why the respondent altered the data, from respondents on 101 engagements. The provisioning of system access was named in 112 answers on 70 engagements, the normalization of data across systems in 77 and the accessibility of data in 55.

“I re-key the SSI details every time the layout shifts between deliveries, we still don’t have working access to the source, so it’s manual. Every time.”Individual Contributor, BPO, Trade, settlement and reconciliation

“We scrub the incoming Data clean every single cycle, as well as that it just turns up dirty again the very next time, so I just spend hours rebuilding same fields week after week”Individual Contributor, ITO, Technology engineering and operations

“Credentials to the client Repository expire and nobody renews them, so I use a workaround login just to keep the files moving… it’s the same thing every status call, we chase it and nothing changes”Individual Contributor, BPO, Corporate and back-office functions

Respondents described data that arrived wrong every cycle and access too limited to fix it where it originated. The correction passed to the delivery team at the last point before the agent used the data.

3.2 Reasons for Altering Financial Institution Processes

The reasons for altering processes were led by decisions that did not come. The speed of the financial institution’s response was named in 152 of the 464 substantive answers on why the respondent altered its processes, from respondents on 94 engagements. Undocumented processes were named in 126 answers on 78 engagements, and the financial institution’s management of its side of the engagement in 61.

“Process never written down anywhere, everyone do it different way. I ask questions and nothing come back so I change step myself to keep work moving.”Individual Contributor, CXM, Corporate and back-office functions

“What they gave us was a slide, not a procedure. And change control cant keep up with the speed we work at so I just adjust it”Individual Contributor, CXM, Corporate and back-office functions

“Last month the Client raise a request that need urgent proccess change but formal Change board only meet monthly, so I make the update myself and inform team lead after”Individual Contributor, KPO, Credit, underwriting and suitability

“The change process was built for quarterly release, not for agent making decision every day… so when servicing rule need small update I please make the same myself rather than wait weeks for formal sign off”Individual Contributor, BPO, Payments, cards and disputes

3.3 Reasons for Altering AI Outputs

The reasons for altering AI outputs were led by output that would not reproduce. The inconsistency of the tools’ outputs given the same input was named in 217 of the 266 substantive answers on why the respondent altered AI outputs, from respondents on 91 engagements. The factual reliability of the output was named in 141 answers on 83 engagements.

“yesterday i ran same question twice and bot gave different policy answer both time, nothing else changed. and i had to correct it before new hire saw wrong one.”Individual Contributor, CXM, Corporate and back-office functions

“Nothing about it is repeatable, i get different Output on same Input twice in one day and Change process assumes normal release cycle not agent running every day”Individual Contributor, KPO, Trade, settlement and reconciliation

A small number of respondents described an instruction not to raise the issue. Across the six written-answer questions on manual alteration, 17 answers from 12 respondents on 10 engagements said the respondent had been told to keep the output moving without escalating the problem.

“The tool gives a different attribution breakdown each time I rerun it… consistency is the one thing it cannot do and my own leadership does not want that raised with the client”Individual Contributor, KPO, Trade, settlement and reconciliation

“It changes its mind between one pass and the next on the same input, this has occured on nearly every run this week, pressure from our own side is only to keep it moving, so I just pick the best version and send”Individual Contributor, ITO, Technology engineering and operations

3.4 Indicators Named in the Reasons

The reasons map onto the indicators reported with the five failure categories. Each reason names a failure that the indicators had rated, and manual alteration is the delivery team’s response downstream of it.

Answers are the substantive written answers to the why question of each method, 400 for data, 464 for processes and 266 for AI outputs. An answer can name more than one indicator.

Indicators of financial institution readiness lead the reasons for altering data and processes, and indicators of AI technology lead the reasons for altering AI outputs. The lowest-rated failure category, financial institution readiness, supplied the leading reasons for two of the three counted methods (Part 2).

Manual alterations were discussed with vendor management and almost never with the financial institution. Across the three questions on internal disclosure, respondents who had altered data, processes or AI outputs answered True in 783 of 1,113 answers (70.35%). Across the three questions on disclosure to the financial institution, they answered True in 13 of 1,004 answers (1.29%).

The constraint on this finding is firm. Disclosure is what respondents said they or their management did. The study records whom the delivery team told and never what the financial institution knew from other sources.

“My own management told me to keep it at working level and not put it in the client summary. So there are two versions of the issue list and neither one is complete.”Engagement Manager, KPO, Corporate and back-office functions

The disclosures to the financial institution that did occur came from engagement managers, 13 of 13. Individual contributors gave 1,175 of the 1,189 answers of True on alteration, and none of their 990 answers of True or False on discussion with the financial institution was True.

4.1 Disclosure to Vendor Management

Internal disclosure was the norm for data and processes and less common for AI outputs. Respondents who had altered data said they discussed it with internal management in 292 of 401 answers (72.82%), and those who had altered processes in 341 of 459 (74.29%). Respondents who had altered AI outputs said so in 150 of 253 answers (59.29%).

Engagements assigned the answer True on internal disclosure had better outcomes on three study measures. For data and processes, they had a lower cancellation rate, a higher engagement indicator mean and lower intent to leave. For AI outputs, the answer was not associated with any study measure at engagement level.

“I revert back the numbers to match the Statement and keep the same in my own tracker… I told my Lead, we never discuss about it with the client.”Individual Contributor, BPO, Trade, settlement and reconciliation

4.1.1 Internal Disclosure of Data Alteration

Respondents who had altered data answered the statement “I discussed the data alteration with internal management.” True was the answer of 292 of the 401 respondents who answered (72.82%), and False of 109 (27.18%). The share did not differ by any segment family after Holm correction.

At engagement level, the answer was associated with cancellation, the engagement indicator mean and intent to leave. Cancellation reached 39.77% of the 88 engagements assigned the answer True, against 64.00% of the 25 assigned False. The association with customer satisfaction was not tested, and the association with the engagement indicator mean did not hold under the Kruskal-Wallis check.

The 210 written responses included 172 positive and 37 negative. The positive responses said the respondent had told the team lead or manager, and some added that the financial institution was not told. The negative responses came from respondents who had told no person, or no person senior to them.

“I dont say to client, only to my Lead… takes time I dont have, and if I pick wrong duplicate nobody would know.”Individual Contributor, BPO, Trade, settlement and reconciliation

“And this happen almost every week, i catch it before it move on. i tell my manager same day, because of that it stop right there, no need to trouble client with it.”Individual Contributor, BPO, Trade, settlement and reconciliation

“This is routine for my team now, we get bad informations every week and nobody fixes the Source, I clean it myself and tell my Lead, not the client”Individual Contributor, KPO, Corporate and back-office functions

4.1.2 Internal Disclosure of Process Alteration

Respondents who had altered processes answered the statement “I discussed the process alteration with internal management.” True was the answer of 341 of the 459 respondents who answered (74.29%), and False of 118 (25.71%). No segment family showed a difference in the share after Holm correction.

Cancellation reached 35.00% of the 100 engagements assigned the answer True, against 65.22% of the 23 assigned False. The engagements assigned True also recorded a higher engagement indicator mean and lower intent to leave, and every check that could be computed held.

The 257 written responses included 225 positive and 29 negative. Respondents described changes reported to the team lead after they were made, and some added that the financial institution was not told.

“Same workaround every week, we do the fix and backfill the paperwork after. And training the next new hire on it is getting old.”Individual Contributor, BPO, Trade, settlement and reconciliation

“Every person on client side do process different way, we chase same point on every status call and nothing move, raising ticket take longer than doing fix myself so most time I just fix it and tell my lead after”Individual Contributor, KPO, Regulatory, audit, tax and legal

4.1.3 Internal Disclosure of AI Output Alteration

Respondents who had altered AI outputs answered the statement “I discussed the AI output alteration with internal management.” True was the answer of 150 of the 253 respondents who answered (59.29%), lower than for data or processes. After Holm correction, the share showed no difference across segment families.

At engagement level, the answer was not associated with any of the four study measures. Two respondent-level clustered checks, for the engagement indicator mean and for intent to leave, reached significance where the engagement-level tests did not.

The 95 written responses included 67 positive and 26 negative. Respondents who told no person described corrections made in silence, some because raising the problem had changed nothing.

“This one i don’t mention to anyone, not even my lead. Output changes between runs and i just pick version that looks right, nobody asks how it happened.”Individual Contributor, KPO, Corporate and back-office functions

“i do not raise this to anyone… it happen most days, i kindly correct the output myself and send it, there is no other option for me”Individual Contributor, ITO, Technology engineering and operations

4.2 Disclosure to the Financial Institution

The three questions on disclosure to the financial institution left nothing to test. Respondents said they or their management had discussed the alteration with the financial institution in 3 of 366 answers on data, 9 of 414 on processes and 1 of 224 on AI outputs. No engagement was assigned the answer True on any of the three, and no association with the study measures could be tested. The absence of disclosure is the finding.

The written answers name the alternative. Among the six written-answer questions on manual alteration, an alteration not told to the financial institution was described in 178 answers from 139 respondents on 73 engagements. An alteration told to the financial institution was described in 30 answers from 25 respondents.

“I fix it quiet. And nobody official know except my team. Client never ask, I do not know if they hear. It happen every week, feels normal but not right.”Individual Contributor, CXM, Corporate and back-office functions

“I do this most days now, my manager know about it, please note we dont tell client, half the records missing values every batch and somebody has to catch that before it break something downstream,”Individual Contributor, ITO, Technology engineering and operations

4.2.1 Disclosure of Data Alteration to the Financial Institution

Respondents who had altered data answered the statement “I or my management discussed data alteration with the client.” True was the answer of 3 of the 366 respondents who answered True or False (0.82%). No engagement was assigned the answer True, and no association could be tested.

The 109 written responses included 96 negative. At least 53 of the 96 said the financial institution was not told, or that the alteration stayed within the vendor team. Eleven said the financial institution had been told, had asked for the corrections or had been shown them.

“When source-system feeds arrive late or malformed, the team logs a data exception, shares the informations with the client’s platform owner, and agrees the correction before it is applied upstream. Nothing touches client data outside that channel.”Portfolio Director, ITO, Technology engineering and operations

“what they did is send us mortgage files that dont reconcile, happens most days at this point, I tell my Lead, we dont tell client,”Individual Contributor, BPO, Credit, underwriting and suitability

4.2.2 Disclosure of Process Alteration to the Financial Institution

Respondents who had altered processes answered the statement “I or my management discussed process alteration with the client.” True was the answer of 9 of the 414 respondents who answered True or False (2.17%). With no engagement assigned the answer True, the associations could not be tested.

The 124 written responses included 99 negative. At least 57 of the 99 said the financial institution was not told, or that the alteration stayed within the vendor team. Twelve said the financial institution had been told, and four described changes signed off by the financial institution before they were applied.

“Same problem, they never gave us a real procedure, just a slide deck, so my team built our own steps, looped the client in as we went”Individual Contributor, BPO, Corporate and back-office functions

“I tell Lead every time, we dont tell client, happens most days, informations we get is never enough to wait on approval, know which call to make most times now”Individual Contributor, CXM, Corporate and back-office functions

4.2.3 Disclosure of AI Output Alteration to the Financial Institution

Respondents who had altered AI outputs answered the statement “I or my management discussed AI output alteration with the client.” A single respondent answered True, of the 224 who answered True or False. The single True answer left no engagement assigned True and nothing to test.

The 50 written responses included 43 negative. At least 22 of the 43 said the financial institution was not told, or that the correction stayed within the vendor team. The single positive response came from a respondent who had told both the financial institution and their own leadership.

“We just patch the breaks ourselves and keep the queue moving, nobody upstairs asks how they got cleared. And I couldn’t tell you if the client ever sees the real touch count.”Individual Contributor, BPO, Trade, settlement and reconciliation

“i tell team lead but not client… this happen maybe once a week, i fix informations in output before it go to review, otherwise sprint slip”Individual Contributor, ITO, Technology engineering and operations

4.3 Sources of the Disclosures That Occurred

Every disclosure to the financial institution recorded in the study came from an engagement manager. Engagement managers answered True on discussion with the financial institution in 13 of their 14 answers, and the study recorded 13 answers of True across the three questions. Individual contributors answered True in none of their 990 answers.

The individual contributors reported 1,175 of the 1,189 True answers on the three counted questions. Their alterations were discussed with internal management in 769 of 1,099 answers (69.97%), and the discussion stopped there. Disclosure to the financial institution was confined to the small number of alterations made or reported at the level of the engagement manager.

Manual alterations were seldom recorded in the written answers, and the reported measures did not show the effort. Across the six written-answer questions on manual alteration, an alteration not logged or logged only in part was described in 138 answers from 115 respondents on 70 engagements. Effort that appeared in no reported measure was described in 26 answers.

“What we settle on in the end is a parallel note. Every synthesis the agents produce, one of my seniors reads it against the source before it goes over, and we hold a sheet of our own recording what changed and why. It is not part of any joint process, the client never sees the sheet.”Engagement Manager, KPO, Corporate and back-office functions

Where a written log of the alterations existed, it was kept by the delivery team. Respondents described a private sheet, a team tracker or a chat thread in place of a system of record, and corrections logged under another heading. A log held by the delivery team is outside the systems of record that the financial institution and its auditors examine.

“the other thing is when the agent got a requirement wrong on a case, and it does, nobody above wanted to say it was ours. it went round for two weeks. my lead said just fix it and log it as data issue. so we did.”Individual Contributor, BPO, Credit, underwriting and suitability

“Some of it makes it into the tracker, plenty does not, this is not every week, maybe once or twice a month, I share informations with my lead when it happen and we move forward”Individual Contributor, KPO, Credit, underwriting and suitability

5.1 Systems of Record

The written answers place any log of manual alteration outside the engagement’s systems of record. The 138 answers that described an alteration not logged, or logged in part, came from 115 respondents on 70 engagements. Respondents described fixes too many to log, logging confined to the large fixes, and changes that left no trace once made.

“i tell my TL but not client, they dont ask anyway, there is no place to put rest of it, so it just disappear off record.”Individual Contributor, CXM, Corporate and back-office functions

“i tell my TL, kindly he say just manage it… we do the same thing every week, i keep no record of the informations i change.”Individual Contributor, CXM, Corporate and back-office functions

“Last week the same paragraph came out different twice in a row on the same draft, consistency is the one thing it cannot do, and I catch it before it reaches an investor every time, that is simply the job and nobody logs it separately”Individual Contributor, CXM, Client servicing and communications

Where a record existed, the delivery team kept it for its own use. Respondents described an internal tracker, a personal note of each sprint’s fixes and a shared sheet, and none was part of a joint process with the financial institution.

5.2 Visibility in Engagement Systems

The engagement’s own reporting did not show the manual alterations. Effort that appeared in no reported measure was described in 26 answers to the manual alteration questions and in 25 answers on how the vendor alters its reporting. Human-corrected work counted as the agent’s was described in 51 answers on reporting.

“no one logs it as a defect, it’s just cleanup we do every time we recieve a bad batch, so it never counts as extra work”Individual Contributor, BPO, Corporate and back-office functions

“I don’t report it as a fix, I just note it in our internal Tracker for the team… my manager sees it but nobody escalates, it just eats time every single week”Individual Contributor, BPO, Corporate and back-office functions

“security approval takes months so we keep reconciling duplicates by hand meanwhile what goes out the other end always looks clean”Individual Contributor, ITO, Technology engineering and operations

The two findings combine. On the respondents’ account, with the alteration unrecorded in the engagement’s systems and the corrected output credited to the agent, the reporting the financial institution receives shows no trace of the alteration (Part 3).

The delivery team named the regulatory and legal obligations that govern the engagements, and a large minority answered that they did not know. The question on which laws and regulations govern the engagement was answered by 795 respondents, and 548 named at least one obligation, on all 150 engagements. The other 247 answered “I do not know” (31.07%), each an individual contributor, on 126 engagements.

The regulatory and legal obligations named cover model risk, anti-money laundering, outsourcing and third-party risk, data protection, operational resilience and conduct. The manual alterations of data, processes, outputs and reporting took place inside engagements governed, on the respondents’ account, by those regulatory and legal obligations.

The study cannot establish that any manual alteration breached a regulatory or legal obligation. The regulatory and legal obligations are reported as the context for an audit. The questions an audit would ask follow from the earlier findings. They concern which alterations were made, whether they were recorded and whom they were reported to.

“the issue is if anyone ever asks how the number was arrived at, the answer involves a decision that i made on a tuesday afternoon because nobody else would.”Individual Contributor, KPO, Corporate and back-office functions

6.1 Obligations Named by the Delivery Team

The respondents named 40 distinct obligations. Model risk governance was named in 219 answers on 58 engagements, “AML and KYC” in 192 on 48, and third-party and outsourcing risk guidance in 189 on 52. A further seven were each named in 173 or 174 answers, including the OCC’s outsourcing guidance, SR 11-7, GDPR, DORA, the EBA outsourcing guidelines and the emerging EU AI Act.

Answers are the 548 answers that named at least one obligation. An answer can name several.

All 150 engagement managers and all 29 portfolio directors who answered the question named at least one obligation. The 247 answers of “I do not know” all came from individual contributors, 40.16% of the 615 individual contributors who answered. Individual contributors were also the role with the largest share answering True on each counted question (Table 21).

6.2 Obligations Governing Financial Institution Material

The regulatory and legal obligations the respondents named fall into classes that concern the material the delivery team altered. Data protection and privacy obligations concern the financial institution’s data. Model risk and AI obligations concern the agents’ outputs and their validation. Outsourcing, third-party and operational resilience obligations concern the processes the vendor operated and the providers it depended on. Conduct, suitability and anti-money laundering obligations concern the decisions the outputs informed.

The study does not assess any obligation and offers no legal conclusion. The classes are set beside the manual alterations as the scope of an audit. An audit would test each class against the alterations the delivery team described, their records and their disclosure.

The written answers record few explicit statements of regulatory exposure. A practice, gap or incident that the respondent linked to a breach or risk of breach of an obligation was described in 16 answers from 14 respondents on 12 engagements. Unclear liability or regulatory accountability for agent errors was described in 83 answers from 80 respondents on 57 engagements.

The delivery team rated its understanding of the governing regulatory and legal obligations higher than the engagement indicator mean, and its training on them near the lowest of any survey question. The respondent’s understanding, management’s understanding and the financial institution’s discussion of the regulatory and legal obligations with the vendor’s team had means of 4.8 to 4.9, on bases of 516 to 801 respondents. The engagement indicator mean of the study set was 4.0 (n = 150 engagements).

The training on the regulatory and legal obligations had a mean of 3.6 (n = 798 respondents), and 385 of the 798 answers (48.25%) were distressed. Its mean ranked 19th lowest of the 146 survey questions with a goodness view. Its association with the engagement indicator mean, r = 0.87 (n = 150 engagements), was the strongest of the four.

“To be fair the hands on agent training was useful, I use it daily. The regulatory side they could not teach us, we learn that from the client’s returns”Individual Contributor, KPO, Regulatory, audit, tax and legal

The ratings of understanding sit beside the answers on which obligations govern the engagement. On that question, 247 of the 795 answers were “I do not know” (31.07%). The rating records what the respondent believes about their own understanding and is not a test of it.

7.1 Regulatory Understanding of the Respondent and Management

The respondent’s understanding and management’s understanding of the governing regulatory and legal obligations were rated alike. The respondent’s understanding had a mean of 4.8 (n = 800 respondents) and was distressed in 228 of 800 answers (28.50%). Management’s understanding had a mean of 4.9 (n = 516 respondents) and was distressed in 149 of 516 answers (28.88%).

Management’s understanding was rated by fewer respondents, 516 against 800. The two ratings were associated with cancellation, customer satisfaction, the engagement indicator mean and intent to leave, without exception under robustness testing.

7.1.1 Respondent Understanding of the Governing Laws and Regulations

Respondents rated the statement “I fully understand how these laws, regulations, and similar affect the outsourcing engagement.” The mean rating was 4.8 and the median 5.0 (n = 800 respondents), and 228 of the 800 answers (28.50%) were distressed. Higher ratings were associated with a higher engagement indicator mean (r = 0.76, n = 150 engagements) and lower intent to leave (r = −0.65, n = 754 respondents). The mean on canceled engagements was 3.5 (n = 304 respondents).

Eight written responses addressed the indicator, too few to support more than description. Seven listed regulatory knowledge learned on the engagement among what the respondent valued. The eighth described a team meeting the regulations for the first time as the tool flagged each case.

“Sharpened my sense of what regulatory accountability actually requires”Individual Contributor, ITO, Financial crime and client onboarding

7.1.2 Management Understanding of the Governing Laws and Regulations

Respondents rated the statement “My management team fully understands these laws, regulations, and similar, and how they affect the engagement,” and 516 answered. The mean rating was 4.9 and the median 5.0 (n = 516 respondents), and 149 of the 516 answers (28.88%) were distressed. Higher ratings were associated with less cancellation (r = −0.53, n = 150 engagements) and lower intent to leave (r = −0.63, n = 492 respondents). Canceled engagements returned a mean of 3.5 across 193 respondents.

No respondent wrote about management’s understanding of the governing obligations.

7.2 Financial Institution Discussion of Obligations With the Delivery Team

One question asked whether the financial institution’s team actively discussed the governing regulatory and legal obligations with the vendor’s team. The answer is the delivery team’s account of that dialogue and is not a measure of what the financial institution held or knew.

7.2.1 Discussion of Obligations Initiated by the Financial Institution

Respondents rated the statement “The customer’s team actively discusses these laws, regulations, and similar with our team.” The mean rating was 4.8 and the median 5.0 (n = 801 respondents), and 231 of the 801 answers (28.84%) were distressed. Higher ratings were associated with a higher engagement indicator mean (r = 0.80, n = 150 engagements) and less cancellation (r = −0.62, n = 150 engagements). Respondents on canceled engagements gave a mean of 3.4 (n = 305 respondents).

No respondent wrote about the financial institution’s discussion of the regulatory and legal obligations.

7.3 Regulatory Training

The training available to the delivery team on the governing regulatory and legal obligations was rated close to the lowest of any survey question. Its mean of 3.6 (n = 798 respondents) ranked 19th lowest of the 146 survey questions with a goodness view.

7.3.1 Regulatory Training That Taught the Tool Rather Than the Regulations

Respondents rated the statement “The training available to me effectively teaches me about these laws, regulations, and similar.” The mean rating was 3.6 and the median 4.0 (n = 798 respondents), and 385 of the 798 answers (48.25%) were distressed. Respondents on canceled engagements gave a mean of 2.4 (n = 308 respondents).

Higher ratings were associated with a higher engagement indicator mean (r = 0.87, n = 150 engagements) and higher customer satisfaction among active engagements (r = 0.70, n = 94 engagements). They were also associated with less cancellation (r = −0.59, n = 150 engagements) and lower intent to leave (r = −0.61, n = 750 respondents).

Five written responses addressed the indicator, all with the strongest negative score, a base of anecdote strength. Each said the employer’s training did not teach the regulations the tasks required, and four said the respondent learned them from a colleague or from the financial institution’s own manual or returns.

“whoever trained us knows nothing about AML, consequently we learn the rules from the client’s own manual.”Individual Contributor, BPO, Financial crime and client onboarding

“tool training was good, nobody taught us the suitability rules”Individual Contributor, CXM, Credit, underwriting and suitability