The Fit Between Governance Processes and the Pace of Agent Work

Reported mismatches between control processes and the pace of agent work were associated with the governance, oversight, and control score in different ways. Forums and change-control cycles showed negative adjusted engagement relationships, while several other timing or capacity patterns did not. The common substantive concern is fit: authority, review, and evidence must operate on the work's timescale. The variation in results requires an account of each mechanism and its limits.

The written accounts describe five broad explanations: slow response by the firms, processes inherited from people-based work, limited experience with agentic AI, output volume and change frequency, and variable output with inadequate evidence. These are respondent explanations rather than identified causes. Each gathers distinct patterns. Their analysis compares respondent scores and relates the engagement share describing a pattern to the engagement score, before and after adjustment. Conditional source populations are stated separately.

Monthly governance forums and change frequency had negative adjusted engagement associations within the process-answer base (partial r = −0.29 and −0.38, n = 133). Their raw associations and conditional respondent contrasts were much less pronounced. Other accounts identified handoff, acceptance, review, documentation, and knowledge gaps, with different or nonsignificant score relationships. This variation limits a single statistical explanation based on pace. It also shows why each reported mechanism requires its own interpretation alongside the broader score.

Monthly forums, reliance on a person noticing errors, and excessive change frequency were classified within answers on manual alteration of processes. Their principal comparisons use the 516 respondents answering that question and the 133 engagements with an answer. Other patterns retain their specified written-answer sources. Table 3.14 reports each pattern's base, respondent comparison, and unadjusted and adjusted engagement association, including null findings.

Nobody gave us the real process so we reconstruct workflow by watching them work… governance forum meets monthly and we work daily so I cant wait for it”

INDIVIDUAL CONTRIBUTOR · CXM, Client servicing and communications

1 Slow Response From the Vendor and the Financial Institution

Decision and response timing connect formal governance with continuing agent work. Joint decision-speed ratings and financial institution response ratings were associated with the wider score, while monthly-forum accounts had a conditional engagement association. Waiting and after-hours escalation accounts supplied smaller and less conclusive evidence. These findings identify different points at which operational response may lag, rather than one verified cause shared by every engagement.

1.1 Governance Forums Meeting Monthly Against Work Moving Daily

Written pattern · Answers on the manual alteration of processes, read under the Part 5 classification of reasons · 44 answers from 41 respondents on 30 engagements

Governance forums that met on a calendar against daily work were associated with lower scores in the adjusted engagement comparison. The substantive concern is a decision cycle that may not match the pace of agent work. Within respondents who answered on process alteration, however, the mean-score contrast was not clearly detected. Those findings use different units and support a qualified account of timing mismatch.

At the engagement level, within the 133 engagements with an answer on process manual alteration, the share describing the pattern was associated with the score at r = −0.06 (p = 0.484). After adjustment for the mean of the other 53 indicators, the association reached partial r = −0.29 (p < 0.001). Across all 150 engagements the adjusted association stood at −0.32 and held between −0.33 and −0.27 when each category's block of the 53 was omitted from the comparison mean. The share describing the pattern was associated with the 53 indicators themselves at only r = −0.08, so the pattern marked weak governance on engagements whose other indicators did not.

The following distributions provide supplementary all-study context and do not replace the eligible process-answer comparisons. The pattern was described by 7.77% (22 of 283) of respondents with a score of 3 or below, 3.60% (19 of 528) above 3 to 7, and none of the 39 above 7. It was described by 6.80% of respondents in Transition/Cut-Over and 6.52% on canceled engagements, against 1.14% in Build-Out and 0.97% in Production. Individual contributors described it at 5.70% (38 of 667) and engagement managers at 2.00% (3 of 150), with no clearly detected difference by role after Holm adjustment. Among individual contributors alone, the 38 who described it stood at 3.18 against 3.91 among the other 629 (cluster-robust p = 0.009).

The 44 answers came from the answers on the manual alteration of processes, where the reading assigned each answer naming a forum, committee, governance body, panel, or approval loop as the reason for the manual alteration, and 23 of them were written on canceled engagements. Respondents described a sign-off forum that met monthly while the work ran daily, and a two-hour change that waited eleven days for a fortnightly forum.

“Decision rights are written down and that part is fine. The issue is timing. A sign-off on the lineage mapping goes to their data governance forum which meets fortnightly, so a two hour change waits eleven days” -Engagement Manager, ITO, Technology engineering and operations

“How I feel is caught in the middle, genuinely, because my team override the early-warning alerts before they escalate them, not for fun, but because the signal needs a human check before anyone trusts it. The decision on which alerts even matter sits with a forum that meets far less than the alerts fire, and what reaches the portfolio director is only what my analysts choose to tell me, far as I know.” -Engagement Manager, KPO, Credit, underwriting and suitability

Part 5 (Manual Alteration of Financial Institution Processes, section 2.4) analyzes the same reading as a reason for process manual alteration.

1.2 Waiting as an Unmeasured Cost

Written pattern · All written answers to the open questions · 10 answers from 10 respondents on 10 engagements

Respondents describing waiting as an unmeasured cost had lower mean scores, although the small pattern did not show a clear engagement-level association. A delay between recognizing a need and obtaining a decision or action can remain absent from performance measures. Ten accounts support a limited description of that issue. They do not establish the extent of waiting among other respondents.

Mean scores were 2.99 for those describing waiting and 4.00 for the remainder. The cluster-robust test on the ten against the other 840 reached p = 0.012. Among individual contributors alone, the five who described waiting stood at 3.36 against 3.87 (cluster-robust p = 0.416).

At the engagement level the share describing waiting was associated with the score at r = −0.09 (p = 0.263, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.07 (p = 0.390).

Waiting was described by 7 of the 283 respondents with a score of 3 or below, 3 of the 528 above 3 to 7, and none of the 39 above 7. Five of the ten were engagement managers, and no segment family differed after Holm adjustment.

The ten answers came from all written answers to the open questions, and 4 of them were written on canceled engagements. Respondents described the days between a change by the financial institution and its arrival on the delivery team, and the weeks between a joint meeting and a decision that held, as costs that appeared in no measure.

“Nothing here asks about the waiting. Client change their conduct rule on a Monday, we hear about it maybe Thursday from a teammate, and everything flagged in between has to be reviewed again by hand. That part never appear anywhere.” -Individual Contributor, CXM, Regulatory, audit, tax and legal

“The joint review side actually worked. Two of their buyers sat with us on the Tuesday call and cleared the flagged POs same day. What no box asks about is the three weeks between one of those calls and any decision that stuck.” -Engagement Manager, CXM, Corporate and back-office functions

1.3 Absence of an Escalation Path for Agent Failure Outside Working Hours

Written pattern · All written answers to the open questions · 14 answers from 14 respondents on 14 engagements

Accounts of missing after-hours escalation paths did not show a clearly detected score relationship. The operational concern remains specific: agent work may continue when the people authorized to respond are unavailable. A broad score may coexist with this reported gap at different levels. The 14 written descriptions support retaining the issue while limiting any conclusion about prevalence or comparative control performance.

The 14 respondents who described it, a small base, stood at 3.91 and the other 836 at 3.99 (cluster-robust p = 0.845). Among individual contributors alone, the 6 who described it stood at 3.85 against 3.86 (cluster-robust p = 0.976).

At the engagement level the share describing it was associated with the score at r = 0.02 (p = 0.808, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.02 (p = 0.787).

The pattern was described by 5 of the 283 respondents with a score of 3 or below, 7 of the 528 above 3 to 7, and 2 of the 39 above 7, a descriptive comparison that does not establish equal prevalence. It was described by 6 individual contributors, 6 engagement managers, and 2 portfolio directors, with no clearly detected difference by role after Holm adjustment, and by 1.82% of respondents in the lowest indicator quintile against 1.88% in the highest.

The 14 answers came from all written answers to the open questions, and 7 of them were written on canceled engagements. Respondents described agents that failed at two in the morning with no person rostered to decide, and a portfolio director on a higher-scoring engagement described the hours each day when only one of the two firms was at work.

“Nothing here asks who we call at 2am when the agent starts closing tickets it shouldn't. There is no such person at our firm, we work it out ourselves.” -Engagement Manager, CXM, Corporate and back-office functions

“The sixes in this section are not a general complaint about how we run the thing together, they come from one narrow gap. Governance in the daytime is genuinely good, and I would say our joint quality loop catches almost everything before it reaches a live endpoint. But the bank's change desk keeps office hours and our desk does not. So the questions about stopping something mid-flight, or holding one view of the risk, are really questions about the eleven hours a day when only one of the two firms is awake. And nobody has written down who makes that call yet. We have not been caught by it, which is luck as much as design.” -Portfolio Director, ITO, Technology engineering and operations

2 Governance Functions Inherited From Before Agentic AI

Processes inherited from people-based delivery were described as poorly fitted to agent work in several distinct ways. Some depended on a person noticing an error, some left knowledge unwritten, and others used service levels, documentation, or handoffs that respondents considered inadequate. Their score relationships varied. Together, these findings locate the shared governance concern in whether controls have been adapted to the work agents perform.

2.1 Change, Risk, Access, and Quality Processes Designed for People

Written pattern · Answers on the manual alteration of processes, read under the Part 5 classification of reasons · 4 answers from 4 respondents on 4 engagements

Four accounts described processes designed for people being carried into agent work without sufficient adaptation. Their governance relevance lies in the fit between operational authority, review, and the pace of work. The evidence is descriptive and too small for the paper's comparative tests. The separately rated joint-model indicator provides related evidence about that fit. One account in each status group does not establish a percentage gradient.

The four respondents recorded a score of 2.79 against 4.00 among the rest, a contrast printed as description only. Three of the four stood with a score of 3 or below and one above 3 to 7, and the four answers came one each from Build-Out, Transition/Cut-Over, Production, and a canceled engagement.

The four accounts were classified within answers on manual alteration of processes. They described an audit workflow unchanged from people-based work and processes written before agents were introduced. The separately rated fit of the joint model to agent work had a positive adjusted cross-category association (partial r = 0.26). That rating supplies related evidence, with no inference that it verifies the mechanism proposed in four accounts.

“This is not one time thing, happen every case here… I tell my lead every time but nothing come back, we just carry on. The audit workflow they gave us was written for people, not for agent, and nobody on their side has looked at it since before the engagement started. So what we do is rebuild the steps as we go and keep own notes. It works, cases close, but the documented process and the actual process are two different things now and everybody knows it and nobody says it out loud. I have kindly asked three times for someone from their side to sit with us for one hour and walk the exceptions. Three times. Nothing. Meanwhile the volumes go up and the sign off window stays the same. My lead is fine, he raise it upward as far as i can tell, but it goes into whatever meeting decides these things and does not come out again. And the frustrating part is when something does go wrong it comes back to us, not to the process, not to the people who wrote it. It comes back to whoever touched it last which is” -Individual Contributor, KPO, Regulatory, audit, tax and legal

“But kindly note this is not a data problem, it is that the process was written before the agent existed. I flag the same issue every review and nothing changes” -Individual Contributor, KPO, Regulatory, audit, tax and legal

Part 5 (Manual Alteration of Financial Institution Processes, section 2.4) analyzes the same reading as a reason for process manual alteration.

2.2 Processes Dependent on a Person Noticing the Error

Written pattern · Answers on the manual alteration of processes, read under the Part 5 classification of reasons · 25 answers from 24 respondents on 19 engagements

Dependence on a person noticing an error was reported within process-alteration accounts without a clearly detected conditional mean-score contrast. The control concern is detection that relies on individual attention rather than an explicit and consistently operating check. The pattern is distinct from automated monitoring and from knowledge that never entered the process. The principal comparison uses eligible process answers, with all-study comparisons presented separately as context.

The contrast within the 516 respondents who wrote an answer on process manual alteration reached cluster-robust p = 0.672. Against all 850 respondents it widened to 3.13 against 4.02. This wider comparison also changes the population, so it does not isolate the contribution of the specific written pattern.

Within the 133 engagements with an answer on process manual alteration, the share describing the pattern was associated with the score at r = −0.10 (p = 0.250), and after adjustment for the mean of the other 53 indicators at partial r = −0.07 (p = 0.400).

The following distributions provide supplementary all-study context and do not replace the eligible process-answer comparisons. The pattern was described by 4.59% (13 of 283) of respondents with a score of 3 or below, 2.08% (11 of 528) above 3 to 7, and none of the 39 above 7, and by 3.03% in the lowest indicator quintile against none in the highest. Twenty-three of the 24 respondents who described it were individual contributors. Among individual contributors alone, the 23 stood at 3.18 against 3.89 among the other 644 (cluster-robust p = 0.009), a contrast drawn against all individual contributors rather than within the base of the reading.

The 25 answers came from the answers on the manual alteration of processes, where the reading assigned each answer naming a check that depended on a person noticing, catching, or spotting the error, and 13 of them were written on canceled engagements. Respondents described workflows documented for an analyst who would notice a comparable that looked wrong, and individual contributors who built manual checks of their own to cover the step the agent skipped.

“their process leans on a person catching the miss, the agent has no such instinct, so i step in myself before the model runs. that is the whole thing really. the workflow they documented assumes an analyst who notices when a comparable looks wrong, and the agent does not notice, it just uses it. so somebody has to be the analyst. that is me most days. i have said in two different forums now that the process needs rewriting for how the tool actually behaves rather than how a person behaves, and both times people agreed and nothing followed. i think the difficulty is that rewriting it means somebody admits the original design was wrong, and the original design has names on it. so instead we have a documented process and a real process and i am the bridge between them. i am not unhappy about it, the work is interesting and i am learning a lot, in all honesty. i just think if i went on leave for two weeks somebody would find out how much of this was being held together by one person noticing things, and” -Individual Contributor, KPO, Corporate and back-office functions

“what they did is build a process that expects a human to catch every mistake before it goes anywhere, but agent doesnt catch anything on its own, it just runs and moves to next case, so i built manual checks in myself to cover for that gap, please note this is not something anyone asked me to do, i just did it because gap was there and somebody had to. their systems never agree with each other on same figures, one say one thing other say another, and nobody ever opened up their api for us properly so we pull flat exports and patch from there instead, which take hours some days, and every access request just goes unanswered for weeks, i dont even chase it anymore because whats the point, i just work around same problem every single time it comes up. if I am honest this whole engagement is closing down soon anyway so i dont know why i am even explaining all this, but i wanted to say it properly at least once, because most days it feels like nobody upstream understands how much of this is held together by me checking things twice that agent should have caught the first time, and i dont discuss about it much with anyone because there isnt really anyone left to discuss about it with at this point, so i just” -Individual Contributor, ITO, Technology engineering and operations

Part 5 (Manual Alteration of Financial Institution Processes, section 2.4) analyzes the same reading as a reason for process manual alteration.

2.3 Checks Retained in Individual Knowledge and Lost to Automation

Written pattern · All written answers to the open questions · 23 answers from 22 respondents on 17 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 4.5

Accounts of checks retained in individual knowledge did not show a clearly detected score relationship. These descriptions identify a governance concern that can be lost when work is automated without making its informal checks explicit. The small written base supports an account of the mechanism respondents described, while the study does not observe the transfer of each check or establish its causal contribution to an outcome.

The overall respondent contrast had p = 0.658. The 22 respondents who described the pattern, a small base, stood at 3.84 and the other 828 at 4.00. Among individual contributors alone, the 18 who described it stood at 3.94 against 3.86 among the other 649 (cluster-robust p = 0.859).

At the engagement level the share describing the pattern was associated with the score at r = −0.04 (p = 0.643, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.01 (p = 0.949).

The pattern was described by 3.18% (9 of 283) of respondents with a score of 3 or below, 2.27% (12 of 528) above 3 to 7, and 1 of the 39 above 7. It was described by 18 individual contributors, 2 engagement managers, and 2 portfolio directors, with no clearly detected difference by role after Holm adjustment, and no segment family differed after Holm adjustment.

The 23 answers came from all written answers to the open questions, and 11 of them were written on canceled engagements. Part 5 counts 14 answers on 9 engagements for the same pattern, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents described a check carried in one person's head and spreadsheet that the agent did not perform and that would leave with that person, and a process that assumed a person would catch the edge cases the agent passed over.

“I patch the Vendor records by hand every single day just to keep the Desk running, I told my manager and hes aware, thats not the issue. The issue is the master data on their side has three sources and they disagree with each other, so depending which one the extract pulls from you get a different supplier address, and the agent cannot tell which is right. I can. Usually. Because Ive seen these vendors for eight months and I know which ones moved office and which ones have two entities. That knowledge is entirely in my head and in a spreadsheet on my machine. Nobody has ever asked me to write it down. If I left tomorrow the Desk would start pushing purchase orders to old addresses within a week and nobody would know why. Ive mentioned that. People nod. Its the sort of thing everyone agrees is a problem and nobody has time to fix because fixing it means somebody has to reconcile three master data sources at a bank, which is a project, not a task. So we carry on. And in truth the work itself is fine, its steady, I just think” -Individual Contributor, CXM, Corporate and back-office functions

“I adjusted the triage step because their process assumes a human catches the edge cases and the agent just doesn't, so I added a manual check before it goes further” -Individual Contributor, ITO, Technology engineering and operations

Part 5 (Leading Dynamics of Manual Alteration, section 4.5) analyzes the pattern as a factor in manual alteration.

2.4 Service Levels Unsuited to Agent Work

Written pattern · All written answers to the open questions · 23 answers from 20 respondents on 15 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 8.4

Service levels unsuited to agent work were described on engagements with somewhat higher observed scores, and the adjusted relationship was not clearly detected. This qualifies a simple reading in which every reported deficiency must accompany a lower composite. The governance question is whether the service commitment measures the pace and form of work now performed. The positive observed association does not establish that the mismatch improves control.

The 20 respondents who described the pattern, a small base, recorded a score of 4.76, against 3.97 among the other 830 (cluster-robust p = 0.107). Among individual contributors alone, the 15 who described it stood at 4.67 against 3.85 (cluster-robust p = 0.140).

At the engagement level the share describing the pattern was associated with the score at r = 0.18 (p = 0.028, n = 150 engagements). After adjustment for the mean of the other 53 indicators the partial r fell to 0.05 (p = 0.555). The adjusted comparison did not detect a clear association beyond the comparison mean.

The pattern was described by 5.00% of respondents in the highest indicator quintile against 0.61% in the lowest, by 3.88% in Production against 1.55% on canceled engagements, and by 1.41% (4 of 283) of respondents with a score of 3 or below against 2.84% (15 of 528) above 3 to 7. No role or segment family differed after Holm adjustment.

The 23 answers came from all written answers to the open questions, and 7 of them were written on canceled engagements. Part 5 counts 4 answers on 4 engagements for the same pattern, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents on active engagements described service level and quality targets written for human pace that lost their meaning once agents handled the work.

Part 5 (Leading Dynamics of Manual Alteration, section 8.4) analyzes the pattern as a factor in manual alteration.

2.5 Documentation Trailing the Build

Written pattern · All written answers to the open questions · 46 answers from 46 respondents on 38 engagements · Listed in Part 5, Additional Conditions Below the Threshold for Statistical Analysis

Documentation lag was described by respondents with higher mean scores, while the adjusted association was not clearly detected. This finding distinguishes reported awareness of a gap from the overall severity of governance weakness. The operational question is whether the record used for oversight reflects the system currently in use. The comparison does not show that better governance causes documentation to lag or that lag is harmless.

The 46 respondents who described it recorded a score of 4.93, against 3.94 among the other 804 (cluster-robust p < 0.001). Among individual contributors alone, the 33 who described it stood at 4.97 against 3.81 (cluster-robust p < 0.001).

At the engagement level the share describing the pattern was associated with the score at r = 0.25 (p = 0.002, n = 150 engagements). After adjustment for the mean of the other 53 indicators the partial r was −0.08 (p = 0.305). The adjusted comparison did not detect a clear association beyond the comparison mean.

The pattern was described by 11.65% of respondents in Production against 2.48% on canceled engagements, and by 9.38% in the highest indicator quintile against 1.82% in the lowest. By the score it was described by 2.12% (6 of 283) of respondents at 3 or below, 6.82% (36 of 528) above 3 to 7, and 4 of the 39 above 7. The share differed after Holm adjustment by class of work and by no other segment family.

The 46 answers came from all written answers to the open questions, and 8 of them were written on canceled engagements. Part 5 lists 2 answers on 2 engagements for the same pattern below its threshold for statistical analysis, among the written answers on the manual alteration methods alone. Documentation that trailed the build was a complaint of engagements that had built enough to document, including the highest-scoring engagement of the study, where an individual contributor named it among the three challenges of the work.

“Last month a servicing rule changed mid week and process still assumed one change a month, besides there was no time allowed to update documentation properly, so exception just get handled by whoever pick it up” -Individual Contributor, BPO, Payments, cards and disputes

2.6 Undefined Handoffs Between the Financial Institution's Own Teams

Written pattern · All written answers to the open questions · 41 answers from 39 respondents on 35 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 3.5

Undefined handoffs within the financial institution were described by respondents with lower mean scores. The control concern is responsibility at the boundary between the institution's own teams, which can remain unresolved even where the vendor's task is clear. The adjusted engagement comparison did not detect a separate association. Lack of mention in the highest quintile records no observed descriptions there, rather than verified absence of the problem.

The 39 respondents who described them recorded a score of 2.81, against 4.05 among the other 811 (cluster-robust p < 0.001). Among individual contributors alone, the 34 who described them stood at 2.89 against 3.92 (cluster-robust p < 0.001).

At the engagement level the share describing the pattern was associated with the score at r = −0.28 (p < 0.001, n = 150 engagements). After adjustment for the mean of the other 53 indicators the partial r was 0.01 (p = 0.876). The adjusted comparison did not detect a clear association beyond the comparison mean.

The pattern was described by 8.13% (23 of 283) of respondents with a score of 3 or below, 3.03% (16 of 528) above 3 to 7, and none of the 39 above 7. It was described by 7.45% of respondents on canceled engagements against 1.94% to 4.00% across the active stages, and by 7.88% in the lowest indicator quintile against none in the highest. The share differed after Holm adjustment by the respondent's understanding of the governing obligations and by regulatory training.

The 41 answers came from all written answers to the open questions, and 25 of them were written on canceled engagements. Part 5 counts 29 answers from 27 respondents on 24 engagements for the same pattern, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents described requirements from two of the financial institution's offices that conflicted with no rule for which prevailed, and discrepancies that had no owner to whom they could be taken.

“If you read one thing here, read this. Nobody on their side owned the surveillance rules. We would get a requirement from compliance in London and a different one from the New York desk for the same scenario, and neither would say which one wins. They answer email fast, Ill give them that, but a fast answer from someone with no authority doesnt move anything. Their lead on this changed twice in eight weeks, so we explained the same design twice to two different people.” -Individual Contributor, ITO, Regulatory, audit, tax and legal

“There is nothing here that the numbers do not already say. I will put it down anyway. When a discrepancy comes out of the anomaly run there is no person to take it to. I asked in the second month, my lead asked also, and the answer was that it depends on the item. That is not an answer. So the practice is, whoever is on the queue that day decides, we write a note in the tracker and move on. Moreover the client side has three teams in the chain and each one says the sign off is with the other, so nothing moves. I have sat in the joint call. It is a status call, nobody makes decision in it. The same items are read out week after week, the same items, and after a while we stopped putting the hard ones on the list because there is no point. On the quality side, both firms count how many exceptions we cleared. Nobody counts how many we cleared wrongly. That is the measure that is missing and I have said this before to my lead. Two firms and neither one owns the number that matters. I do not think anybody will read this box. The engagement is finished anyway…” -Individual Contributor, KPO, Regulatory, audit, tax and legal

Part 5 (Leading Dynamics of Manual Alteration, section 3.5) analyzes the pattern as a factor in manual alteration.

3 The Newness of Agentic AI to Both Firms

Learning, knowledge transfer, and accountability were described as unresolved features of agentic delivery across a range of scores. These patterns concern preparation for unfamiliar work and how experience is brought into the operating model. Their inconsistent statistical relationships limit a single explanation based on newness. Each pattern nonetheless identifies a distinct responsibility relevant to oversight.

3.1 Practice Developed on the Live Engagement

Written pattern · All written answers to the open questions · 95 answers from 93 respondents on 71 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 4.3

Developing practice during live delivery was described across a range of scores without a clearly detected contrast. This identifies uncertainty in how the two firms learn to govern unfamiliar work. The governance question is whether authority, checks, and evidence develop alongside live practice or are settled beforehand. The written accounts describe that learning context, without measuring the effectiveness or causal consequences of learning on the engagement.

The overall respondent contrast had p = 0.199. The 93 respondents who described it stood at 3.76 and the other 757 at 4.02. Among individual contributors alone, the 67 who described it stood at 3.54 against 3.90 (cluster-robust p = 0.082).

At the engagement level the share describing the pattern was associated with the score at r = −0.12 (p = 0.154, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.15 (p = 0.065).

The pattern was described by 13.07% (37 of 283) of respondents with a score of 3 or below, 10.04% (53 of 528) above 3 to 7, and 3 of the 39 above 7. It was described by 8.74% to 10.80% of respondents across the active stages and 12.73% on canceled engagements, and by 10.04% of individual contributors, 14.67% of engagement managers, and 4 of the 31 portfolio directors, with no clearly detected difference by role after Holm adjustment. No segment family differed after Holm adjustment.

The 95 answers came from all written answers to the open questions, and 42 of them were written on canceled engagements. Part 5 counts 12 answers from 11 respondents on 10 engagements for the same pattern, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents described delivery teams that had never assured agent output at volume and invented their own checking, and teams that learned to run agents on the financial institution's live work.

“The playbook is fine on paper. We were never shown how to assure agent output at this volume, so we invented our own checking. Pricing is not solved either, whatever the deck says… Leadership is far from the floor. That is all.” -Engagement Manager, BPO, Trade, settlement and reconciliation

“These ones are for my desk, not the firm. Nobody here had run agents live before us, we learned on the client's tax memos. Elsewhere i am told it is better.” -Individual Contributor, KPO, Regulatory, audit, tax and legal

Part 5 (Leading Dynamics of Manual Alteration, section 4.3) analyzes the pattern as a factor in manual alteration.

3.2 Knowledge Transfer Not Completed

Written pattern · All written answers to the open questions · 25 answers from 24 respondents on 22 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 4.4

Incomplete knowledge transfer was described without a clearly detected score relationship. Its relevance to control is the transfer of the operational knowledge needed to recognize exceptions, interpret output, and respond to problems. The small written base supports keeping the issue distinct from general experience or training. The result does not establish that knowledge transfer was equally complete across the remaining engagements.

The overall respondent contrast had p = 0.241. The 24 respondents who described it, a small base, stood at 3.64 and the other 826 at 4.00. Among individual contributors alone, the 21 who described it stood at 3.71 against 3.87 (cluster-robust p = 0.637).

At the engagement level the share describing the pattern was associated with the score at r = −0.10 (p = 0.202, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = 0.09 (p = 0.250).

The pattern was described by 3.53% (10 of 283) of respondents with a score of 3 or below, 2.65% (14 of 528) above 3 to 7, and none of the 39 above 7, and by 4.24% in the lowest indicator quintile against 0.62% in the highest. No role or segment family differed after Holm adjustment.

The 25 answers came from all written answers to the open questions, and 12 of them were written on canceled engagements. Part 5 counts 6 answers on 5 engagements for the same pattern, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents described knowledge transfer measured by the number of sessions held rather than by what was captured, and manual fixes by a former team that had never been written down.

“SME knowledge-transfer completion was tracked by the number of sessions held rather than whether the underwriting logic was actually captured, so the metric looked complete while real institutional knowledge kept leaving with departing staff.” -Individual Contributor, BPO, Payments, cards and disputes

“Nobody wrote down which bits the old team fixed by hand, we are rediscovering it case by case” -Individual Contributor, CXM, Client servicing and communications

Part 5 (Leading Dynamics of Manual Alteration, section 4.4) analyzes the pattern as a factor in manual alteration.

3.3 Unclear Liability and Regulatory Accountability

Written pattern · All written answers to the open questions · 83 answers from 80 respondents on 57 engagements · Listed in Part 5, Additional Conditions Below the Threshold for Statistical Analysis

Unclear liability and regulatory accountability were described across a range of scores. These accounts concern who is responsible when agent work causes an error, a different question from routine workflow ownership or a respondent's knowledge of governing obligations. The descriptions identify uncertainty rather than adjudicate liability. Their concentration in Production is a cross-sectional observation, with no inference about when accountability became unclear.

The 80 respondents who raised it recorded a score of 4.32, against 3.96 among the other 770 (cluster-robust p = 0.134). Among individual contributors alone, the 58 who raised it stood at 4.21 against 3.83 (cluster-robust p = 0.138).

At the engagement level the share raising it was associated with the score at r = 0.09 (p = 0.275, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.06 (p = 0.485).

Unclear liability was raised by 9 of the 39 respondents with the score above 7 (23.08%), against 8.13% (23 of 283) at 3 or below and 9.09% (48 of 528) above 3 to 7. It was raised by 14.56% of respondents in Production, against 9.14% to 9.60% in the earlier stages and 7.76% on canceled engagements, and by 11.25% in the highest indicator quintile against 9.70% in the lowest. The share differed after Holm adjustment by class of work.

The 83 answers came from all written answers to the open questions, and 26 of them were written on canceled engagements. Part 5 lists 2 answers on 2 engagements for the same pattern below its threshold for statistical analysis, among the written answers on the manual alteration methods alone. The question of who answered for an agent's error reached the highest-scoring engagements and those in Production. An individual contributor whose own score stood above 7, and who credited the engagement's staffing and training, was not sure who would answer for a costly agent error.

“On my last account we wait three weeks for one AI person to be assigned, here I get a specialist on the same day I ask. The training also is better now, they teach us the payments side properly. Only thing, if an agent makes a costly mistake I am not sure who stands up for it, we never discuss about that” -Individual Contributor, CXM, Corporate and back-office functions

3.4 Comparison With Prior Engagements

Written pattern · All written answers to the open questions · 237 answers from 206 respondents on 117 engagements · Listed in Part 5, Additional Conditions Below the Threshold for Statistical Analysis

Respondents comparing the engagement with prior work had higher observed mean scores, while the adjusted association was not clearly detected. Their comparisons provide context for how they assess current control arrangements. They do not create a measured before-and-after study. The accounts describe respondents' frames of reference and do not establish that prior experience improved the present engagement's controls.

The 206 respondents who made a comparison recorded a score of 4.35, against 3.88 among the other 644 (cluster-robust p = 0.002). Among individual contributors alone, the 142 who made one stood at 4.09 against 3.80 (cluster-robust p = 0.108).

At the engagement level the share making a comparison was associated with the score at r = 0.24 (p = 0.003, n = 150 engagements). After adjustment for the mean of the other 53 indicators the partial r was 0.00 (p = 0.974).

Comparison was made by 36.88% of respondents in the highest indicator quintile against 21.82% in the lowest, and by 38.46% (15 of 39) of respondents with the score above 7 against 21.91% (62 of 283) at 3 or below. It differed by role after Holm adjustment, made by 24 of the 31 portfolio directors (77.42%) against 26.67% of engagement managers and 21.29% of individual contributors, and portfolio directors recorded the highest the score of the three roles. It also differed by the manual alteration methods used and by disclosure to the financial institution.

The 237 answers came from all written answers to the open questions, and 89 of them were written on canceled engagements. Part 5 lists 1 answer on 1 engagement for the same pattern below its threshold for statistical analysis. The comparisons ran in both directions. On the highest-scoring engagement of the study, an individual contributor compared a change agreed and written down the same day with a weekly call on an earlier account, while on a canceled engagement an individual contributor recalled a previous project that had a playbook.

“On my last engagement any change had to wait for a weekly call, here the two sides agree a change same day and it is written down properly. Issues still go into two seperate trackers and we chase both, that part is not solved. Overall it runs better than what I saw before” -Individual Contributor, KPO, Credit, underwriting and suitability

“Felt behind from day one. My last project had an actual playbook. Here you learn it by watching the person next to you, half of them are guessing too. But nobody above us has shipped one of these before, so i don't know who'd write it.” -Individual Contributor, CXM, Client servicing and communications

4 Volume and Speed of AI Output Beyond the Controls

Review volume and change frequency describe two different capacity demands on control. Review-capacity ratings were associated with the wider score, while the written volume pattern was less clearly differentiated by the composite. Change frequency showed a stronger adjusted engagement association than its raw association. The relevant governance question is the fit between work and control capacity, with model dependence stated explicitly.

4.1 Human Review Overrun by Agent Volume

Written pattern · Answers scored negative on human review capacity, read in full · 123 answers from 115 respondents on 80 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 6.3

Human review overwhelmed by agent volume was described across a range of composite scores. The operational concern is whether review capacity can examine the work it is expected to control. The written source consists of answers classified negative on that capacity, so the pattern is not an independent survey of every respondent's review workload. A high composite does not establish that the reported capacity gap was otherwise contained.

The 115 respondents who described it stood at 4.13 and the other 735 at 3.97 (cluster-robust p = 0.409). Among individual contributors alone, the 93 who described it stood at 3.93 against 3.85 (cluster-robust p = 0.667).

At the engagement level the share describing the pattern was associated with the score at r = 0.07 (p = 0.369, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = 0.00 (p = 0.954).

The pattern was described by 12.01% (34 of 283) of respondents with a score of 3 or below, 14.58% (77 of 528) above 3 to 7, and 4 of the 39 above 7. It was described by 17.14% of respondents in Build-Out, falling to 13.59% in Production and 11.18% on canceled engagements, and by 12.73% in the lowest indicator quintile against 13.12% in the highest. No role or segment family differed after Holm adjustment.

The 123 answers are the written answers scored negative on human review capacity, each read in full, and 122 of them state that review was overrun by the volume of agent output or by its own capacity. Eight of them also describe review reduced to a tick, a click-through, or a small sample, too few to stand as a pattern of their own. Thirty-six of the 123 were written on canceled engagements. On one of the five engagements without a single distressed governance rating, an individual contributor reported that the agents produced more test scripts in a morning than the team could read.

“The part that reaches me is volume of review. Agents produce more test scripts in a morning than we can properly look at, so my day becomes reading output instead of building anything. i dont mind it exactly, team is good and my lead shields us where he can. Still i finish the week and cant say what i actually built. That is new for me, in addition nobody planned for it, it just arrived with the tooling.” -Individual Contributor, ITO, Technology engineering and operations

The rating of review capacity is analyzed under Co-Management and Governance Across the Other Four Categories, and Part 5 (Leading Dynamics of Manual Alteration, section 6.3) reports its association with manual alteration.

4.2 Change Frequency Beyond the Capacity of Change Control

Written pattern · Answers on the manual alteration of processes, read under the Part 5 classification of reasons · 101 answers from 93 respondents on 65 engagements

Change frequency exceeding change-control capacity had a negative adjusted association with the score in the eligible engagement comparison. The substantive concern is a process whose operating cycle cannot accommodate the changes it is meant to govern. The raw engagement correlation was small, and the respondent contrast was not clearly detected within process-answer respondents. The stronger conditional estimate depends on the specified comparison model.

Within the 133 engagements with an answer on process manual alteration, the share describing the pattern was associated with the score at r = −0.08 (p = 0.366). After adjustment for the mean of the other 53 indicators the association reached partial r = −0.38 (p < 0.001). Across all 150 engagements the adjusted association stood at −0.45 and held between −0.49 and −0.40 when each category's block of the 53 was omitted from the comparison mean. The share describing the pattern was associated with the 53 indicators themselves at r = −0.17.

The following distributions provide supplementary all-study context and do not replace the eligible process-answer comparisons. The pattern was described by 18.73% (53 of 283) of respondents with a score of 3 or below, 7.58% (40 of 528) above 3 to 7, and none of the 39 above 7. It was described by 14.56% of respondents in Production and 13.66% on canceled engagements, against 6.86% in Build-Out, and by 16.36% in the lowest indicator quintile against 8.75% in the highest. It differed by role after Holm adjustment, written by 92 individual contributors and one engagement manager. Among individual contributors alone, the 92 who described it stood at 3.13 against 3.98 among the other 575 (cluster-robust p < 0.001).

The 101 answers came from the answers on the manual alteration of processes, where the reading assigned each answer naming a change process, change board, queue, window, or release cycle that could not keep pace as the reason for the manual alteration, and 45 of them were written on canceled engagements. The share describing it differed after Holm adjustment by role, by management's understanding of the governing obligations, by the manual alteration methods used, and by disclosure to the financial institution. Respondents described a change process that assumed one change a month against several a day, and a queue of pending changes longer than the queue of work.

“The Change Process assumes one change a month and we make several a day… we cannot wait for a ruling so we rule ourselves on most exceptions” -Individual Contributor, BPO, Corporate and back-office functions

“Angry if im honest. The queue of pending changes is longer than the queue of actual work, and the decision arrive after the deadline it was meant to help.” -Individual Contributor, ITO, Client servicing and communications

Part 5 (Manual Alteration of Financial Institution Processes, section 2.4) analyzes the same reading as a reason for process manual alteration.

5 Non-Deterministic Output and Missing Logs

Variable output and limited evidence create different assurance problems. Output that cannot be signed off and absence of a quality assurance method concern acceptance, while missing logs concern reconstruction and diagnosis. Their score relationships varied. The joint logs-and-consistency comparison describes the lowest mean score among respondents rating both poorly, with no test of causal interaction.

5.1 Output That Cannot Be Signed Off

Written pattern · All written answers to the open questions · 29 answers from 27 respondents on 23 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 5.4

Output that could not be signed off was described by a small group with low mean scores. This connects acceptance of agent work with the wider control environment: delivery cannot be treated as fully assured where responsible staff cannot establish a basis for sign-off. The adjusted engagement association was not clearly detected. The finding supports a specific reported assurance problem, not a causal severity ranking among written patterns.

The 27 respondents who described it, a small base, recorded a score of 2.34, against 4.05 among the other 823 (cluster-robust p < 0.001). Among individual contributors alone, the 26 who described it stood at 2.22 against 3.93 (cluster-robust p < 0.001).

At the engagement level the share describing the pattern was associated with the score at r = −0.32 (p < 0.001, n = 150 engagements). After adjustment for the mean of the other 53 indicators the partial r was −0.11 (p = 0.180). The adjusted comparison did not detect a clear association beyond the comparison mean.

The pattern was described by 7.77% (22 of 283) of respondents with a score of 3 or below, 0.95% (5 of 528) above 3 to 7, and none of the 39 above 7. It was described by 5.28% of respondents on canceled engagements and by none in Production, and by 5.45% in the lowest indicator quintile against none in the highest. The share differed after Holm adjustment by indicator quintile, by the manual alteration methods used, and by intent to leave.

The 29 answers came from all written answers to the open questions, and 18 of them were written on canceled engagements. Part 5 counts 28 answers from 26 respondents on 22 engagements for the same pattern, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents described output that gave a different figure on every run and a delivery team that chose among the runs, the best of three or the middle value, before submitting one.

“We get duplicates, have to guess which is real, quiet from them we take as a yes, won't sign off on output that changes each rerun, pick one and go” -Individual Contributor, BPO, Credit, underwriting and suitability

“the problem is we cannot sign off on something that will not reproduce, so i pick the best run out of three and that's the one we submit” -Individual Contributor, ITO, Payments, cards and disputes

Part 5 (Leading Dynamics of Manual Alteration, section 5.4) analyzes the pattern as a factor in manual alteration.

5.2 Absence of a Quality Assurance Method for Variable Output

Written pattern · Answers scored negative on the vendor's quality assurance for variable output · 126 answers from 116 respondents on 81 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 6.1

Absence of a quality assurance method for variable output was associated with lower respondent mean scores. Quality assurance needs a defensible basis for accepting outputs that differ across runs. The pattern comes from answers classified negative on that capability, with a less conclusive engagement-level association. It should remain distinct from the tools' testability rating and from the separate evidence provided by logs.

The 116 respondents describing the pattern had a mean score of 3.60, compared with 4.05 for the other 734 (cluster-robust p = 0.010). Among individual contributors alone, the 93 who described it stood at 3.39 against 3.94 among the other 574 (cluster-robust p = 0.005).

At the engagement level the share describing the pattern was associated with the score at r = −0.16 (p = 0.055, n = 150 engagements), and after adjustment for the mean of the other 53 indicators at partial r = −0.11 (p = 0.178).

The pattern was described by 18.73% (53 of 283) of respondents with a score of 3 or below, 10.80% (57 of 528) above 3 to 7, and 6 of the 39 above 7, so it reached the highest-scoring respondents as well. It was described by 16.77% of respondents on canceled engagements against 9.71% to 13.20% across the active stages, and by 17.58% in the lowest indicator quintile against 11.25% in the highest. The share differed after Holm adjustment by intent to leave and by no other segment family.

The 126 answers are the written answers scored negative on the vendor's quality assurance for output that differs between runs, and 54 of them were written on canceled engagements. Twenty-six of them also described output that could not be signed off. Respondents described individual contributors who had become the quality assurance step, and a screenshot of every case as the only check on a tool that never gave the same answer twice.

“There is no QA for a tool that never give the same answer twice. I screenshot every case before I submit it. That is my QA. Nobody ask me to do it” -Individual Contributor, BPO, Financial crime and client onboarding

“My low marks come down to staffing, not to the leadership. We were promised two analysts with market research background and got two juniors who had never read a fund factsheet. Then QA. Nobody at my level was ever shown how to check work that comes out different every run, we werent given a method, we invented one ourselves.” -Engagement Manager, KPO, Corporate and back-office functions

The rating of the vendor's quality assurance for variable output is reported in Part 2 (Vendor Capability, section 6.2), and Part 5 (Leading Dynamics of Manual Alteration, section 6.1) reports its association with manual alteration.

5.3 Missing Logs and Evidence of Agent Actions

Rating crossing · Logs, traces, and usage telemetry against consistency of output across runs · 750 respondents who rated both

Respondents giving low ratings to both logs and output consistency had lower mean scores than those giving a low rating to one or neither. Limited logs can make the reasons for variable results difficult to reconstruct. The comparison is descriptive and does not establish statistical interaction or causal compounding. The full composite in this comparison includes the logs item.

Among the 750 respondents with both numeric answers, 184 rated both items at 1–3 and had a mean score of 2.06. The 114 rating exactly one item at 1–3 had a mean of 3.12. The 452 rating neither at 1–3 had a mean of 5.00. The 100 respondents missing at least one answer are excluded. Across engagements, the two indicator ratings correlated at r = 0.94 (n = 150).

The ordered means describe joint reported weakness, with no test of interaction or causal compounding. The composite in this comparison includes the logs and telemetry item, so part of its relationship with the groups is built into the score. This is not independent confirmation of the logs rating. The logs item itself had an engagement-level correlation of 0.66 with the score within canceled engagements.

Written accounts concerning these limitations are examined alongside the relevant controls. Logs and telemetry drew 101 written answers and explanation at audit depth drew 163, both reported under Technical Controls and Governance Across the Engagement, and output that could not be signed off is the written form of the same failure. Part 2 (AI Technology, sections 2.3 and 8.3) reports the two ratings, and Part 5 (Leading Dynamics of Manual Alteration, sections 5.2 and 6.6) their association with manual alteration.

6 Respondent Explanations Locating Problems Upstream

Some respondents located the problem upstream of the indicator they were rating. These accounts connect a local control assessment with decisions or dependencies elsewhere in the engagement. They did not show a clearly detected adjusted score association. Their interpretive value lies in identifying the proposed location of responsibility, not in establishing the upstream account as a verified root cause.

“i felt there is no point giving different marks… it all come back to one thing, their compliance side will not confirm the risk rating threshold for us, therefore every case sit open. the individual people are fine, on top of that they are polite about it, but nobody will put a decision in writing” -Individual Contributor, CXM, Financial crime and client onboarding