Practices Adopted in Place of Controls

Human rework and normalized workarounds were associated with lower governance, oversight, and control scores, while habit in place of an explicit control showed no clearly detected gradient. These practices connect delivery continuity with work that formal control and reporting may represent incompletely. Their differing relationships with the composite matter: a specific unwritten dependency can be reported even where the broader score is relatively favorable.

These practices distinguish correction of agent work from its visibility in the record used for oversight. Human rework and normalized workarounds had lower observed scores, while unreported effort showed a weaker adjusted relationship and habit showed no clearly detected gradient. Four of the 39 respondents above score 7 described habit in place of a control. The normalized-workaround partial correlation of −0.18 is nominally significant and remains subject to the stated test-family qualification. Table 3.18 retains each practice and the separately identified proposed remedies.

Theres a model defect form youre supposed to file when the Bot gets a fix wrong. Nobody fills it out, takes about twenty minutes and it goes to a queue that answers in like three weeks. So my team keeps a notepad file in Teams with what actually worked on which ticket type, and we just use that.”

INDIVIDUAL CONTRIBUTOR · ITO, Technology engineering and operations

1 Workarounds Adopted as Standard Practice

The four practices distinguish human correction, routine workarounds, unreported effort, and dependence on habit. Rework and missing effort had negative observed relationships with the score, while normalized workarounds retained a nominal adjusted association. Habit showed no clearly detected relationship. Each contributes a different view of how actual work can diverge from the process visible to oversight.

1.1 Human Rework Absorbing Agent Failure

Written pattern · All written answers to the open questions · 344 answers from 244 respondents on 111 engagements · Analyzed in Part 5, Manual Alteration of AI Outputs, section 2.1

Human rework absorbing agent failure was associated with lower mean scores and a strong negative observed engagement association. Operational delivery can depend on human correction whose contribution is incompletely represented in formal control and reporting. The adjusted association was weaker and not clearly detected. This distinguishes substantial reported rework from evidence of an independently identified governance effect.

The 244 respondents who described it recorded a score of 2.88, against 4.44 among the other 606 (cluster-robust p < 0.001). Among individual contributors alone, the 232 who described it stood at 2.87, against 4.40 among the other 435 (cluster-robust p < 0.001).

At the engagement level the share describing the practice was associated with the score at r = −0.57 (95% CI −0.67 to −0.45, p < 0.001, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.11 (p = 0.180).

The practice was described by 52.30% (148 of 283) of respondents with a score of 3 or below, 17.80% (94 of 528) above 3 to 7, and 2 of the 39 above 7. It was described by 44.41% of respondents on canceled engagements, against 16.00% in Build-Out, 19.60% in Transition/Cut-Over, and 23.30% in Production, and by 51.52% in the lowest indicator quintile against 10.00% in the highest. Individual contributors described it at 34.78% (232 of 667), engagement managers at 7.33% (11 of 150), and 1 of the 31 portfolio directors, a difference by role that held after Holm adjustment. The practice differed after Holm adjustment in 11 of the 19 segment families and in none of the structural families.

Rework followed the capacity of human review. Among respondents who rated human review capacity against agent volume 1 to 3, 43.57% (166 of 381) described rework, against 14.90% (59 of 396) of those who rated it 4 to 7 and 1 of the 28 who rated it 8 to 10 (chi-square = 87.67, p < 0.001, Cramér's V = 0.33).

The 344 answers came from all written answers to the open questions, and 207 of them were written on canceled engagements. Respondents described a funding cutoff missed twice and reconciled manually while a rule change waited for approval, and documents checked manually where no test environment had proved the agent before live work began.

“A funding rule change sat with them for four days… And in that time the agent kept releasing on the old rule, so we missed the cutoff twice and i squared it by hand after. Nobody could tell me who was allowed to stop it. Moreover when they asked later what the agent did, we had nothing to show them.” -Individual Contributor, BPO, Trade, settlement and reconciliation

“there is no test setup, we could not prove anything before it ran on live cases. passport and address proof it reads sometimes, everything else i check by hand” -Individual Contributor, CXM, Financial crime and client onboarding

Part 5 (Manual Alteration of AI Outputs, section 2.1) analyzes human rework within the answers on the manual alteration of AI outputs.

1.2 Workarounds Normalized Into the Process

Written pattern · All written answers to the open questions · 152 answers from 137 respondents on 86 engagements · Analyzed in Part 5, Manual Alteration of Financial Institution Processes, section 2.4

Workarounds normalized into the process were associated with lower scores and a negative adjusted estimate. When an exceptional correction becomes routine, its visibility and formal control may not develop with it. The nominal adjusted result requires the stated multiple-testing qualification. It supports retaining the pattern as a distinct analytical subject without treating its significance as settled under every test family.

The 137 respondents who described them recorded a score of 3.16, against 4.15 among the other 713 (cluster-robust p < 0.001). Among individual contributors alone, the 130 who described them stood at 3.15, against 4.04 among the other 537 (cluster-robust p < 0.001).

At the engagement level the share describing the practice was associated with the score at r = −0.36 (95% CI −0.50 to −0.22, p < 0.001, n = 150 engagements). After adjustment for the mean of the other 53 indicators the adjusted estimate was partial r = −0.18 (p = 0.027), one of three written patterns of the 41 to keep an adjusted association.

The practice was described by 24.24% of respondents in the lowest indicator quintile and 5.00% in the highest. It was described by 25.44% (72 of 283) of respondents with a score of 3 or below, 12.31% (65 of 528) above 3 to 7, and none of the 39 above 7. It was described by 21.43% of respondents on canceled engagements, against 12.57% to 13.59% across the active stages, a difference that did not survive Holm adjustment. Individual contributors described it at 19.49% (130 of 667), engagement managers at 4.67% (7 of 150), and no portfolio director did, a difference by role that held after Holm adjustment.

The 152 answers came from all written answers to the open questions, and 75 of them were written on canceled engagements. Respondents described workarounds that were never signed off and that became normal with time, and silence from the financial institution taken as approval.

“my team does this most days now, in addition nobody signs off the workaround, it just becomes normal after time” -Individual Contributor, BPO, Trade, settlement and reconciliation

“This happens every week, when i raise it with Client they just revert back with nothing useful, so silence is treated as approval by us now” -Individual Contributor, KPO, Trade, settlement and reconciliation

Part 5 (Manual Alteration of Financial Institution Processes, section 2.4) analyzes the workaround within the answers on the manual alteration of processes.

1.3 Effort Absent From Reported Figures

Written pattern · All written answers to the open questions · 71 answers from 64 respondents on 54 engagements · Analyzed in Part 5, Disclosure of Manual Alteration, section 3.3

Effort absent from reported figures was described by respondents with lower mean scores. This connects the record used for oversight with the work required to sustain delivery. The engagement association was negative, while the adjusted analysis did not detect a clear separate association. The reported visibility gap remains substantively relevant, while the adjusted result limits a claim of a separate statistical association.

The 64 respondents who described it recorded a score of 3.14, against 4.06 among the other 786 (cluster-robust p < 0.001). Among individual contributors alone, the 50 who described it stood at 3.07, against 3.93 among the other 617 (cluster-robust p = 0.002).

At the engagement level the share describing the practice was associated with the score at r = −0.27 (95% CI −0.41 to −0.11, p < 0.001, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.14 (p = 0.098).

The practice differed across engagement status, described by 2.86% of respondents in Build-Out, 6.00% in Transition/Cut-Over, 7.77% in Production, and 11.18% on canceled engagements. It was described by 11.31% (32 of 283) of respondents with a score of 3 or below, 6.06% (32 of 528) above 3 to 7, and none of the 39 above 7. Engagement managers described it at 9.33% (14 of 150) and individual contributors at 7.50% (50 of 667), with no clearly detected difference by role after Holm adjustment. After Holm adjustment it differed by indicator quintile, engagement status, cancellation risk band, the financial institution's discussion of the obligations, process family, the count of identified manual alteration methods, and disclosure to the financial institution.

The 71 answers came from all written answers to the open questions, and 41 of them were written on canceled engagements. An engagement manager described two nights of manual re-tagging on work no count recorded, and an individual contributor described errors caught before release that appeared in no count.

“Incident last cycle: we pull month of call recordings and half have no adviser ID or call reason on them, so my team spends two nights re-tagging by hand. People are willing on client side, thats not issue. But I am the one explaining to our own leadership why throughput is short, and my QA leads are burning out on work nobody counts.” -Engagement Manager, CXM, Regulatory, audit, tax and legal

“problem is nobody counts the wrong answers we caught, only ones the client complains about” -Individual Contributor, CXM, Client servicing and communications

Part 5 (Disclosure of Manual Alteration, section 3.3) analyzes effort left out of the reported numbers within the answers on the three counted methods.

1.4 Habit in Place of a Control

Written pattern · All written answers to the open questions · 91 answers from 87 respondents on 69 engagements · Analyzed in Part 5, Disclosure of Manual Alteration, section 3.4

Habit standing in for an explicit control was described across score bands without a clearly detected score gradient. This is a distinct governance finding because favorable composite ratings can coexist with work that depends on particular people and unwritten routines. The analysis does not formally establish that the score fails as a detection instrument. It shows that this reported practice was not sharply distinguished by the measured comparisons.

The 87 respondents who described it recorded a score of 3.93, against 4.00 among the other 763 (cluster-robust p = 0.705). Among individual contributors alone, the 64 who described it stood at 3.65, against 3.89 among the other 603 (cluster-robust p = 0.297).

At the engagement level the share describing the practice was associated with the score at r = −0.02 (p = 0.789, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.01 (p = 0.872).

The practice was described by 11.31% (32 of 283) of respondents with a score of 3 or below, 9.66% (51 of 528) above 3 to 7, and 4 of the 39 above 7. It was described by 11.52% in the lowest indicator quintile and 8.75% in the highest, and by 7.77% to 12.40% across engagement status. Portfolio directors described it at 19.35% (6 of 31), engagement managers at 11.33% (17 of 150), and individual contributors at 9.60% (64 of 667), and no segment family differed after Holm adjustment.

The rating about replacing informal fixes with explicit checks did not show a clear association with the written habit pattern. The pattern was described by 27 of 267 respondents rating that item at 1–3, 47 of 459 at 4–7, and 10 of 70 at 8–10. The corresponding shares were 10.11%, 10.24%, and 14.29%. The ordinary chi-square result was 1.14 (p = 0.567, Cramér's V = 0.04), subject to the stated clustering limitation. This is not a formal validation test of the rating's detection capability.

The 91 answers came from all written answers to the open questions, and 36 of them were written on canceled engagements. An individual contributor who wrote that the providers gave notice of change described a colleague who read every release note unasked, a step written into no process. Another individual contributor described an engagement held together by two people talking directly, an arrangement that stopped when one of them was away for a week.

“Feel fine about this part mostly… the providers do tell us in advance when something change, that works. What these questions leave out is the person side. One colleague reads every release note from the model provider and posts a summary in our channel. He started it himself, nobody asked him to. It is not written into any process anywhere.” -Individual Contributor, ITO, Technology engineering and operations

“what actually held this together was my lead and one guy on their side talking directly, everything else on paper never got used… that is not a process, when my lead was out for a week nothing moved at all. none of these questions get at how much of it was just two people knowing each other” -Individual Contributor, CXM, Client servicing and communications

Part 5 (Disclosure of Manual Alteration, section 3.4) analyzes workarounds held as personal habit within the answers on the three counted methods.

2 Remedies Proposed by Respondents

The proposed remedies addressed operational responsibility, shared evidence, and timely authority. They included named people able to stop an agent, one incident record, written limits on autonomous action, and ownership of input definitions. These proposals connect the paper's reported weaknesses with possible control responses. They are suggestions from respondents, not interventions whose effectiveness the study evaluated. Remedies appeared in 225 answers from 203 respondents on 118 engagements, including 95 answers on canceled engagements.

Remedies were proposed by 27.56% (78 of 283) of respondents with a score of 3 or below, 21.02% (111 of 528) above 3 to 7, and 35.90% (14 of 39) above 7, the highest share of any score band. Portfolio directors proposed them at 41.94% (13 of 31), engagement managers at 26.00% (39 of 150), and individual contributors at 22.64% (151 of 667), and no segment family differed after Holm adjustment.

Of the 225 answers, 142 were judged negative on at least one governance indicator, and the indicators named the control the remedy would restore. Co-management indicators appeared in 56 of the 142 and financial institution readiness indicators in 54, against 20 for AI technology, 18 for the third-party ecosystem, and 6 for vendor capability. The indicators named most often were the financial institution's speed of response to issues raised (19), its provisioning of access and identities for agents (18), the effectiveness of its management of its side (16), the maturity of its data governance (12), and the placement of human review points (12). The remedies asked for one person on each side with authority to stop an agent, one shared list of incidents in place of two, a written statement of what an agent may do without a person, and one owner on the financial institution's side for the definitions the agents read.

“How I feel is nobody can press stop. What would help is simple, one person on each side with authority to pause the run, and the check should sit before the payment file goes out not after” -Individual Contributor, BPO, Corporate and back-office functions

“two separate queues is the real issue. one shared incident list would fix most of it, at present their soc raise their own ticket and we raise ours and nobody reconciles them.” -Engagement Manager, ITO, Technology engineering and operations

“Practically, one page is needed: which program parameters an agent may set without a human, agreed by client treasury and by our operations together. Until that is signed, our reviewers approve everything on trust and the review point is decoration” -Engagement Manager, BPO, Payments, cards and disputes

“I pulled the field mappings myself, the same policy field is named three ways across their systems and nothing lines up. What would help is a single owner over there for the definitions… right now we ask four people and get four answers.” -Individual Contributor, ITO, Client servicing and communications

Taken together, the findings place the operation and visibility of controls at the center of the governance problem. Lower scores were associated with canceled engagements and substantial manual correction, while specific reported weaknesses also appeared among more favorable composite assessments. Co-management showed consistent conditional associations, but category composition and model sensitivity limit claims of statistical uniqueness. The evidence supports examining who can decide, stop, review, and reconstruct agent work across organizational boundaries. It does not establish causal effects or certify effective governance at a numeric threshold. The respondents' remedies address those operating responsibilities and remain proposals to be evaluated in practice.

Part 5 (Leading Dynamics of Manual Alteration, section 10) analyzes the remedies proposed by the delivery team within the answers on manual alteration.