Source: https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/co-management/
Author: Phil Hatch
Publisher: Akholi

Co-Management

The 2026 Report · Part 2

# Co-Management

Joint decisions, risk management, change control, human review, and agent management

By Phil Hatch · Akholi · Version 1.8.2

Complete study · PDFPart 2 · PDFStudy overview / Part 2 / Co-Management

Distressed ratings across co-management identified concerns about the two firms' shared decisions, coordination, and review arrangements. Co-management is the category of indicators that exist only because two firms share the engagement. Its 20 indicators describe how the financial institution and the vendor governed the engagement together, managed its risks and issues, and controlled changes to the agents. They also describe how the two firms reviewed agent output and adapted their joint structures to the work performed by agents. The category had a mean of 3.65 across the 150 engagements, second lowest of the five, and 7,866 of its 15,968 answers (49.26%) were distressed. Its indicators are reported in the order of the survey section from which they come.

Table 2.4. The indicators of co-management, from the lowest mean rating

| Indicator | Mean rating | Distressed answers | Distressed share |
| --- | --- | --- | --- |
| 5.4 Readiness of the Joint Model for Expected Agent Autonomy | 3.60 | 407 of 808 | 50.37% |
| 4.5 Quality Measures Calibrated to Human Rather Than Agent Error | 3.61 | 405 of 792 | 51.14% |
| 1.2 Unsettled Boundary of Agent Autonomy | 3.61 | 406 of 808 | 50.25% |
| 2.4 Authority to Stop a Misbehaving Agent | 3.62 | 406 of 804 | 50.50% |
| 3.2 Agent Updates Disclosed to the Other Firm After the Fact | 3.62 | 394 of 803 | 49.07% |
| 5.3 Joint Governance Model Designed for Work Performed by People | 3.63 | 395 of 803 | 49.19% |
| 2.1 Separate Risk Records Maintained by the Two Firms | 3.64 | 383 of 793 | 48.30% |
| 4.1 Placement of Human Review Points Across the Two Firms | 3.64 | 393 of 797 | 49.31% |
| 4.3 Routine Approval of Agent Output at Human Review Points | 3.65 | 393 of 794 | 49.50% |
| 2.3 Resolution Speed Against the Spread of Agent Error | 3.65 | 400 of 809 | 49.44% |
| 1.1 Co-Management Structures That Existed on Paper | 3.66 | 385 of 789 | 48.80% |
| 1.4 Accountability for Problems Arising From Agent Action Dissolved Across the Two Firms | 3.66 | 402 of 799 | 50.31% |
| 3.3 Unmonitored Agent Drift Between Releases | 3.66 | 388 of 806 | 48.14% |
| 1.3 Unowned Handoffs in Shared Workflows | 3.67 | 388 of 808 | 48.02% |
| 4.2 Human Review Capacity Against Agent Output Volume | 3.67 | 381 of 805 | 47.33% |
| 3.1 Joint Change Control Outpaced by Agent Behavior | 3.68 | 399 of 798 | 50.00% |
| 4.4 Quality Checks Designed for Human Rather Than Agent Error | 3.68 | 377 of 787 | 47.90% |
| 5.1 Agent Actions Visible Only Inside Each Firm's Own Tools | 3.68 | 392 of 780 | 50.26% |
| 5.2 Joint Decision Speed Mismatched to Agent Speed | 3.69 | 381 of 786 | 48.47% |
| 2.2 Issue Management Through Separate Lists | 3.73 | 391 of 799 | 48.94% |

Mean ratings in Table 2.4 are computed across respondents. The category figure of 3.65 is computed across engagements.

The category differs from the others in two respects. Respondents on the 56 canceled engagements rated it lower than any other category, at a mean of 2.22 across its indicators (Table 2.2). Among active engagements it was rated lower by respondents on the 20 engagements in Production (4.07) than on the 31 in Build-Out (4.67), the widest difference between stages of any category. Its indicators were associated with intent to leave as closely as any in the study, from r = −0.69 to r = −0.64. Their associations with customer satisfaction among active engagements lay in the lowest range of any category, from r = 0.59 to r = 0.63 (n = 94 engagements each).

Written accounts described weaknesses in joint decision-making, shared records, review capacity, and change coordination. Some respondents considered the arrangements insufficiently adapted from earlier staffed delivery. The open question on co-management received comments from 551 of 850 respondents (64.82%). Both favorable and unfavorable experiences are reported below.

> “The joint calls work, but I am the one holding the gap between an agent that acts in seconds and a decision that comes on Thursday. That sits with me personally, and my leads know to call me at any hour when the queue backs up.” -Engagement Manager, CXM, Corporate and back-office functions

> “It is that neither side see what the other changed until the allocation is already posted... for that reason we find out at month end, every month” -Individual Contributor, BPO, Trade, settlement and reconciliation

Some respondents described acting before approval or outside formal procedures. Part 3 examines the implications of these reported practices for control design and verification.

> We had a weekly joint call and a shared issue list from week one, which is more than I usually get at this stage. The gap was on review. Three of us were approving routing decisions we did not have time to actually look at, and the volume was still small. But we never got far enough for any of it to be tested properly. Accountability was clear on paper at least” ENGAGEMENT MANAGER · CXM, Technology engineering and operations

## 1 Joint Governance and Decision Rights

Accounts of joint governance contrasted working decision forums with unresolved autonomy limits, workflow ownership, and responsibility for remediation. Four indicators describe how the two firms managed the engagement together and settled who decides. They cover the effectiveness of co-management, the boundary of agent autonomy, the ownership of each step in a shared workflow, and the accountability for problems arising from agent action. Each had about half of its answers in the distressed range, from 388 of 808 answers (48.02%) for the ownership of workflow steps to 406 of 808 (50.25%) for the boundary of autonomy. Across the four, every association with the core study measures held under the applicable robustness checks.

Negative accounts described inactive forums, unresolved autonomy limits, unclear workflow ownership, and disputes over remediation. Positive accounts also described joint structures that met regularly and resolved issues. The presence of a forum does not by itself establish whether its individual governance functions operate effectively.

> “What it costs us is Fridays. Because it has never been settled which policy questions the agent may answer alone, my leads sit in a two-hour joint review every week clearing a backlog of held tickets that mostly should never have been held. The team notices, and it is the part of the week they resent.” -Engagement Manager, CXM, Corporate and back-office functions

> “The problem is nobody can tell me who is allowed to switch the thing off. In February the tool post same vendor exception into queue forty times, maybe more, and I ring my lead, my lead ring somebody else, and by then it is next morning already.” -Individual Contributor, BPO, Corporate and back-office functions

### 1.1 Effectiveness of Co-Management Structures

Indicator C136 · Rate how effectively the customer and vendor jointly manage this engagement now that agents perform part of the work · Full statistical analysis ›

Accounts of co-management distinguished joint arrangements that resolved decisions from meetings and separate records that left delivery teams without effective support. The mean rating was 3.7 and the median 4.0 (n = 789 respondents). Respondents on canceled engagements rated co-management lower than respondents on active engagements in any stage (mean 2.2, n = 299 respondents).

Figure 2.23. Effectiveness of Co-Management Structures: associations with the study measures

Scroll horizontally to view the complete figure.

Effectiveness of co-management structures: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.61 [−0.70, −0.50] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.62 [0.48, 0.73] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.84 [0.78, 0.88] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.67 [−0.72, −0.60] 739 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70297780789999997 | 1 |
| 95% confidence interval | 1 | -0.50041016940000005 | 1 |
| Reported association | 0 | -0.61162177169999998 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.47983637600000001 | 1 |
| 95% confidence interval | 1 | 0.73231049339999998 | 1 |
| Reported association | 0 | 0.62198235609999997 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.78306686420000005 | 1 |
| 95% confidence interval | 1 | 0.88017878149999995 | 1 |
| Reported association | 0 | 0.83814497539999999 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72373393659999996 | 1 |
| 95% confidence interval | 1 | -0.59940877520000002 | 1 |
| Reported association | 0 | -0.66612441600000005 | 1 |

Note: Survey question: “Rate how effectively the customer and vendor jointly manage this engagement now that agents perform part of the work”.

Distressed ratings: 385 of 789 answers (48.80%).

Group means and numeric-answer bases: Individual contributor (n = 607): 3.54. Engagement manager (n = 150): 3.84. Portfolio director (n = 30): 5.00. Build-Out (n = 165): 4.73. Transition/Cut-Over (n = 232): 4.58. Production (n = 93): 4.02. Canceled (n = 299): 2.23.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Co-management of the engagement by the two firms was associated with every core study measure, without a single association overturned in the applicable robustness checks. Engagements that rated co-management higher had a lower cancellation rate (r = −0.61, 95% CI −0.70 to −0.50, n = 150 engagements). Customer satisfaction among active engagements was higher where co-management was rated higher (r = 0.62, 95% CI 0.48 to 0.73, n = 94 engagements). Effective co-management was associated with a higher engagement indicator mean, computed without the joint-management indicator (r = 0.84, 95% CI 0.78 to 0.88, n = 150 engagements). Intent to leave was lower among respondents who rated co-management higher (r = −0.67, cluster bootstrap 95% CI −0.72 to −0.60, n = 739 respondents). Distressed ratings came from 385 of the 789 answers (48.80%), and the engagement mean was distressed on 64 of the 150 engagements.

The written responses addressing co-management numbered 209, from 199 respondents on 115 engagements. They were more evenly divided than for most indicators, with 127 of the 209 responses negative (60.77%) and 79 of the 209 responses positive (37.80%). The negative responses described co-management that did not exist, or that met without deciding anything, while the engagement proceeded through separate trackers and private records. The positive responses described joint calls that worked and placed the remaining weakness in the review step or in the means to stop an agent.

> “There was a joint operating forum on paper — I have the terms of reference still, names on it, a weekly slot — and in eight months it met four times, twice with only my side in the room. So the ones are not a judgment on the people, they are a judgment on a structure that existed only in a document.” -Engagement Manager, CXM, Corporate and back-office functions

> “Two firms is generous, it is whoever picks up the exception queue that morning deciding what a stalled contract exception does next, and whatever we decide is what happened. Nobody above either side ever looks at it.” -Individual Contributor, BPO, Corporate and back-office functions

> “We had a finding on duplicate vendor payments sitting for eleven days, nobody would say if it was theirs to sign or ours. In the end my lead just approved it himself, and the review after that was one person clicking through forty items in an hour.” -Individual Contributor, KPO, Regulatory, audit, tax and legal

> “On my previous ITO account we have a joint bridge for every P2 and both sides sit on it. Here the client incident manager joins only when it is already a P1, so the middle cases drift for a day or two.” -Engagement Manager, ITO, Technology engineering and operations

> “my last account was pension admin, one weekly call. here there is a joint call every morning, my lead sits in it and sometimes an answer comes back same day. that is better than what i had before, even if none of it reaches my queue” -Individual Contributor, KPO, Client servicing and communications

Negative joint-management comments were associated with higher cancellation and lower customer satisfaction. More negative respondent sentiment was associated with higher intent to leave. These associations do not establish direction. The written accounts identify both functional arrangements and cases in which operational personnel made decisions without effective joint support.

### 1.2 Unsettled Boundary of Agent Autonomy

Indicator C137 · Rate how clearly the two firms have settled who decides what agents may do without human approval · Full statistical analysis ›

The boundary between permitted agent action and required human approval remained unresolved in several written accounts. The mean rating was 3.6 and the median 3.0 (n = 808 respondents). Respondents on canceled engagements rated the settlement of autonomy lower than respondents on active engagements in any stage (mean 2.2, n = 309 respondents).

Figure 2.24. Unsettled Boundary of Agent Autonomy: associations with the study measures

Scroll horizontally to view the complete figure.

Unsettled boundary of agent autonomy: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.51] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.59 [0.44, 0.71] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.84 [0.79, 0.88] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.67 [−0.73, −0.60] 758 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71095774329999994 | 1 |
| 95% confidence interval | 1 | -0.51227373379999996 | 1 |
| Reported association | 0 | -0.62151062349999997 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.43685236710000003 | 1 |
| 95% confidence interval | 1 | 0.70604553260000003 | 1 |
| Reported association | 0 | 0.58747017270000002 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.78684406210000002 | 1 |
| 95% confidence interval | 1 | 0.88237662839999997 | 1 |
| Reported association | 0 | 0.8410486546 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72769896619999996 | 1 |
| 95% confidence interval | 1 | -0.60205247780000004 | 1 |
| Reported association | 0 | -0.66925420369999999 | 1 |

Note: Survey question: “Rate how clearly the two firms have settled who decides what agents may do without human approval”.

Distressed ratings: 406 of 808 answers (50.25%).

Group means and numeric-answer bases: Individual contributor (n = 625): 3.48. Engagement manager (n = 150): 3.91. Portfolio director (n = 31): 4.81. Build-Out (n = 163): 4.66. Transition/Cut-Over (n = 240): 4.56. Production (n = 96): 4.12. Canceled (n = 309): 2.16.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

The settlement of agent autonomy between the two firms was associated with cancellation, customer satisfaction, the engagement indicator mean, and intent to leave, and no robustness check overturned an association. Cancellation was less common where the boundary of autonomy was rated settled (r = −0.62, 95% CI −0.71 to −0.51, n = 150 engagements). Among active engagements, customer satisfaction was higher where the boundary of autonomy was settled (r = 0.59, 95% CI 0.44 to 0.71, n = 94 engagements). The engagement indicator mean, computed without the autonomy indicator, was higher where the boundary was settled (r = 0.84, 95% CI 0.79 to 0.88, n = 150 engagements). Respondents who rated the boundary settled reported lower intent to leave (r = −0.67, cluster bootstrap 95% CI −0.73 to −0.60, n = 758 respondents). Distressed ratings came from 406 of the 808 answers (50.25%), and the engagement mean was distressed on 68 of the 150 engagements.

Autonomy-boundary comments numbered 48, from 48 respondents on 42 engagements. 34 were negative. Half described an absent or unresolved agreement on actions requiring human approval. Some respondents proposed a single agreed list. Favorable accounts described using such an agreement. Its reported existence is distinct from evidence that it was consistently enforced.

> “Give us checklist for what the agent is allowed to write without review, because right now every one of us decide by ourself.” -Individual Contributor, CXM, Client servicing and communications

> “We set the decision rights in the first two weeks and wrote them down, that part has held up. Anything the agent does above a sev-3 goes to a named person on the bank side and a named person on ours. Where it is thinner is review depth.” -Engagement Manager, ITO, Technology engineering and operations

> “In my last job on the fraud queue there is one name against every rule, if it break you know who to call. Here nobody know who says yes when the agent close an alert by itself. My Lead ask them, they ask us back, and we keep triaging by hand in the middle.” -Individual Contributor, ITO, Technology engineering and operations

> “At present every material disposition still comes back to a person on our side, which is safe but it does not scale, and we have not had the conversation with their compliance leadership about which alert types the agent may close on its own. That conversation has been deferred twice now.” -Portfolio Director, BPO, Financial crime and client onboarding

> “Decision rights are clear, we know what the agent can file without a person signing off.” -Individual Contributor, BPO, Corporate and back-office functions

No significant association was detected between autonomy-boundary sentiment and the study measures. The accounts concern definition and implementation of approval requirements. They do not measure net time savings from automation or demonstrate that a written list alone provides effective control.

### 1.3 Unowned Handoffs in Shared Workflows

Indicator C138 · Rate how clearly ownership of each workflow step is assigned when agents, vendor staff, and customer staff work in the same process · Full statistical analysis ›

Unassigned ownership of workflow steps left some cases waiting while the two firms disputed responsibility for the handoff. The mean rating was 3.7 and the median 4.0 (n = 808 respondents). The assignment of step ownership was rated lower on canceled engagements than on active engagements in any stage (mean 2.2, n = 305 respondents).

Figure 2.25. Unowned Handoffs in Shared Workflows: associations with the study measures

Scroll horizontally to view the complete figure.

Unowned handoffs in shared workflows: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.65 [−0.73, −0.55] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.61 [0.47, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.69 [−0.74, −0.62] 758 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.73490452760000002 | 1 |
| 95% confidence interval | 1 | -0.54833588300000002 | 1 |
| Reported association | 0 | -0.65135793340000003 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.46668358859999998 | 1 |
| 95% confidence interval | 1 | 0.72435258729999996 | 1 |
| Reported association | 0 | 0.61148020089999999 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.80183133719999999 | 1 |
| 95% confidence interval | 1 | 0.89105641840000005 | 1 |
| Reported association | 0 | 0.85253925539999997 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.74398356139999999 | 1 |
| 95% confidence interval | 1 | -0.62442737540000004 | 1 |
| Reported association | 0 | -0.68850748340000001 | 1 |

Note: Survey question: “Rate how clearly ownership of each workflow step is assigned when agents, vendor staff, and customer staff work in the same process”.

Distressed ratings: 388 of 808 answers (48.02%).

Group means and numeric-answer bases: Individual contributor (n = 626): 3.54. Engagement manager (n = 150): 3.89. Portfolio director (n = 31): 5.26. Build-Out (n = 165): 4.77. Transition/Cut-Over (n = 239): 4.58. Production (n = 99): 4.24. Canceled (n = 305): 2.18.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Every association of workflow-step ownership with the four core study measures held, in each of the applicable robustness checks. Engagements that rated the ownership of each workflow step higher had a lower cancellation rate (r = −0.65, 95% CI −0.73 to −0.55, n = 150 engagements). Customer satisfaction among their active engagements was higher (r = 0.61, 95% CI 0.47 to 0.72, n = 94 engagements). Clear ownership of each step was associated with a higher engagement indicator mean, computed without the ownership indicator (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated step ownership higher reported lower intent to leave (r = −0.69, cluster bootstrap 95% CI −0.74 to −0.62, n = 758 respondents). After Holm correction, which limits false-positive conclusions within a specified family of comparisons, the rating also differed by process family. Distressed ratings came from 388 of the 808 answers (48.02%), and the engagement mean was distressed on 64 of the 150 engagements.

The written responses addressing the ownership of workflow steps numbered 82, from 81 respondents on 58 engagements, and 70 of the 82 were negative. Among the 68 responses in the comment field on co-management, 40 described a step in the shared process that belonged to neither firm. When the agent or a person handled that step wrongly, each firm said it belonged to the other, and the case waited. The remedy respondents proposed was a name against each step.

> “When the assistant sends a HR case to wrong queue and it sits four days, we have meeting about it and both sides explain why it is not their part. And then next week same thing. My lead says we cannot change the routing rules ourself because it is joint decision, but joint decision takes longer than the SLA it is breaking.” -Individual Contributor, CXM, Corporate and back-office functions

> “A single attribution rule change sat for eleven weeks … Word came back it was our call to make, our lead said it was theirs, and in the meantime the agent kept producing breakdowns on the old basis.” -Individual Contributor, KPO, Trade, settlement and reconciliation

> “Nobody owned the step where a generated test case gets accepted, we assumed their QA lead did, he assumed we did, and it sat there. Also every decision we raised took three weeks to revert back, which is not a pace anything can be built at.” -Engagement Manager, ITO, Technology engineering and operations

> “Last month close we get 400 exceptions in one night and only two of us to review them. But nobody decide who signs off the ones the agent already cleared. i check them all myself, hence we finish late” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “We keep our own sheet now. Every morning my senior analyst lists which summaries the agent touched overnight and who on their side signs each one, and we walk through it on a fifteen minute call before their clinical lead starts. It is not governance. But it is the only place both firms see the same list.” -Engagement Manager, KPO, Credit, underwriting and suitability

> “Overrides on NAV breaks sit for days because sign off belong to nobody, not the client, not us, and my team clear them anyway so the pack go out on time.” -Engagement Manager, BPO, Trade, settlement and reconciliation

Negative comments on workflow ownership were associated with higher cancellation, and more negative respondent sentiment with higher intent to leave. Respondents described delayed cases and disputed responsibility where ownership was unclear. These are reported experiences rather than a verified causal sequence.

### 1.4 Allocation of Accountability Between the Firms

Indicator C139 · Rate how clearly accountability lands on one firm when an agent’s action causes a problem in this engagement · Full statistical analysis ›

Respondents described delays in assigning responsibility between the two firms after problems arising from agent action. The mean was 3.7 and the median 3.0 (n = 799 respondents). The mean on canceled engagements was 2.2 (n = 299), lower than in each active stage.

Figure 2.26. Allocation of Accountability Between the Firms: associations with the study measures

Scroll horizontally to view the complete figure.

Allocation of accountability between the firms: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.64 [−0.73, −0.54] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.61 [0.46, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.87 [0.82, 0.90] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.68 [−0.73, −0.61] 750 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72693671360000001 | 1 |
| 95% confidence interval | 1 | -0.53625939659999999 | 1 |
| Reported association | 0 | -0.64139805220000001 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.46301249690000001 | 1 |
| 95% confidence interval | 1 | 0.7221191213 | 1 |
| Reported association | 0 | 0.60853979550000004 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.82062498009999996 | 1 |
| 95% confidence interval | 1 | 0.9018491958 | 1 |
| Reported association | 0 | 0.866879222 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.73154043160000004 | 1 |
| 95% confidence interval | 1 | -0.61167985020000004 | 1 |
| Reported association | 0 | -0.67626006240000003 | 1 |

Note: Survey question: “Rate how clearly accountability lands on one firm when an agent’s action causes a problem in this engagement”.

Distressed ratings: 402 of 799 answers (50.31%).

Group means and numeric-answer bases: Individual contributor (n = 616): 3.54. Engagement manager (n = 150): 3.86. Portfolio director (n = 31): 4.94. Build-Out (n = 164): 4.70. Transition/Cut-Over (n = 239): 4.64. Production (n = 97): 4.05. Canceled (n = 299): 2.18.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Accountability for problems arising from agent action showed an association with each of the four core study measures that survived all applicable robustness checks. Clearer accountability was associated with less cancellation (r = −0.64, 95% CI −0.73 to −0.54, n = 150 engagements). Among active engagements, customer satisfaction was higher where accountability was rated clearer (r = 0.61, 95% CI 0.46 to 0.72, n = 94 engagements). The engagement indicator mean, computed without the accountability indicator, was closely associated with it (r = 0.87, 95% CI 0.82 to 0.90, n = 150 engagements). Intent to leave was lower among respondents who rated accountability clearer (r = −0.68, cluster bootstrap 95% CI −0.73 to −0.61, n = 750 respondents). Distressed ratings came from 402 of the 799 answers (50.31%), and the engagement mean was distressed on 65 of the 150 engagements.

The written responses addressing accountability numbered 51, from 50 respondents on 37 engagements, and 45 of the 51 were negative. When a problem arose from an agent's action, respondents described days spent deciding which firm was at fault before either fixed it, and incident records that named a symptom in place of an owner. Several said that stopping the agent was quick and that deciding who owned the harm was not.

> “both firms are perfectly able to describe what went wrong and neither is willing to write down who carries it. We have a joint issue log, on paper. In practice the log records symptoms. When a suite the agent generated missed a defect that later surfaced in UAT, the entry reads environment instability, which is true enough, and also convenient for everyone in the room, ourselves included.” -Portfolio Director, ITO, Technology engineering and operations

> “There was a letter that went out with the wrong deal date in it last quarter and eleven days went by before anyone would say where the fault sat, and in the end no one did.” -Individual Contributor, CXM, Client servicing and communications

> “escalation is the whole issue here. when the agent dispositions an alert wrong nobody owns it after. my side says compliance owns disposition, compliance says we built the rule. so it sits in queue for weeks and then ages out. which is why the backlog number they report is meaningless.” -Individual Contributor, ITO, Financial crime and client onboarding

> “when the assistant mis-reads an underwriting requirement and a new-business file sits for a week, there is nobody on our side who carries it. I raise it, my manager raises it, and it comes back as a question for the client.” -Engagement Manager, BPO, Credit, underwriting and suitability

> “nobody owns errors when the agent ships something broken” -Engagement Manager, ITO, Payments, cards and disputes

No significant association was detected between accountability sentiment and the study measures. Respondents described disputes over investigation and remediation after incidents. The accounts suggest unresolved responsibility in those cases, without establishing the contractual allocation or legal liability of either firm.

> The incident report attributed the failed release to a 'process gap' rather than naming the specific agent action that triggered it, which let the dashboard show a process metric instead of the accountability question everyone actually wanted answered.” INDIVIDUAL CONTRIBUTOR · ITO, Technology engineering and operations

## 2 Shared Risk and Issue Management

Separate records and unresolved authority were reported to impede coordinated responses to agent-related risks and operational problems. Four indicators describe how the two firms managed the risks and issues that agent work produced. They ask whether the firms kept one view of agent risk and managed issues as one team. The indicators also ask whether the firms resolved issues at the speed at which agent error spreads and could stop an agent behaving badly in live operation. Each had about half of its answers in the distressed range, from 383 of 793 answers (48.30%) for the shared view of risk to 406 of 804 (50.50%) for the authority to stop an agent. All four held their associations with the core study measures under the applicable robustness checks.

Shared risk management concerns prospective threats. Issue management concerns problems that have occurred. Both require coordinated records, ownership, and follow-up. Respondents described fragmented records and delays in obtaining authority to suspend an agent. The subsections distinguish record coordination, resolution speed, and suspension capability.

> “The change control part is honestly fine, we see their updates before they land and they see ours... you should ask about the stopping instead. If the agent starts pushing rubbish into the pipeline on a Friday night, I do not know who I ring, my lead says it is their side, they say it is ours.” -Individual Contributor, ITO, Technology engineering and operations

Some accounts described automated processing continuing while an incident remained unresolved. The relevant control question is whether the firms can detect the problem and suspend affected processing promptly. Neither human nor automated errors are inherently confined to a single action.

### 2.1 Separate Risk Records Maintained by the Two Firms

Indicator C140 · Rate how well the two firms maintain a single shared view of the risks that agent work introduces · Full statistical analysis ›

Separate, unreconciled risk lists were a recurring concern in accounts of the two firms' shared view of agent-related risks. The mean rating was 3.6 and the median 4.0 (n = 793 respondents). Respondents on canceled engagements rated the shared view of risk lower than respondents on active engagements in any stage (mean 2.2, n = 297 respondents).

Figure 2.27. Separate Risk Records Maintained by the Two Firms: associations with the study measures

Scroll horizontally to view the complete figure.

Separate risk records maintained by the two firms: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.51] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.62 [0.48, 0.73] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.67 [−0.73, −0.61] 744 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71159983449999997 | 1 |
| 95% confidence interval | 1 | -0.51323162170000003 | 1 |
| Reported association | 0 | -0.62230755319999997 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.48215652720000002 | 1 |
| 95% confidence interval | 1 | 0.733707149 | 1 |
| Reported association | 0 | 0.62382964490000004 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.79827310480000002 | 1 |
| 95% confidence interval | 1 | 0.88900158009999997 | 1 |
| Reported association | 0 | 0.8498156227 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.73077105090000005 | 1 |
| 95% confidence interval | 1 | -0.60884653740000005 | 1 |
| Reported association | 0 | -0.67360545959999996 | 1 |

Note: Survey question: “Rate how well the two firms maintain a single shared view of the risks that agent work introduces”.

Distressed ratings: 383 of 793 answers (48.30%).

Group means and numeric-answer bases: Individual contributor (n = 612): 3.50. Engagement manager (n = 150): 3.92. Portfolio director (n = 29): 5.00. Build-Out (n = 164): 4.65. Transition/Cut-Over (n = 236): 4.57. Production (n = 96): 4.03. Canceled (n = 297): 2.21.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

The applicable robustness checks left intact each association between a shared view of agent risk and the four core study measures. Engagements where the two firms kept a shared view of risk had a lower cancellation rate (r = −0.62, 95% CI −0.71 to −0.51, n = 150 engagements). A shared view of risk was associated with higher customer satisfaction among the active engagements (r = 0.62, 95% CI 0.48 to 0.73, n = 94 engagements). The engagement indicator mean, computed without the risk-view indicator, was higher where the view was shared (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated the shared view higher reported lower intent to leave (r = −0.67, cluster bootstrap 95% CI −0.73 to −0.61, n = 744 respondents). Distressed ratings came from 383 of the 793 answers (48.30%), and the engagement mean was distressed on 65 of the 150 engagements.

Shared-risk comments numbered 26, all supplied in the joint-management field. 19 were negative. Sixteen described separate risk lists that were not reconciled, including duplicate discussion without clear resolution. Positive accounts described a jointly reviewed log.

> “What did not exist, and would have to next time, is one risk log rather than two, and some way for each side to see what the other's agents had actually done on a ticket without having to ask … None of that is hard. It just needs doing before day one rather than at month two.” -Portfolio Director, CXM, Corporate and back-office functions

> “same for the shared view of risk. two firms, two lists, nobody put them side by side.” -Individual Contributor, ITO, Financial crime and client onboarding

> “What is missing is a common risk list. Their side keeps their own, we keep ours, and QA only catches a bad suggestion in the pricing code when somebody remembers to compare them.” -Individual Contributor, ITO, Technology engineering and operations

> “Both sides keep one risk log for the agents, this works well. our Lead meets their dispute team every week and issues get closed in that call.” -Individual Contributor, BPO, Payments, cards and disputes

More negative risk-management sentiment was associated with higher cancellation and intent to leave. Separate records may impede coordination, but their existence alone does not establish that either firm lacked knowledge of a risk. The study did not examine other communication channels or risk systems.

### 2.2 Issue Management Through Separate Lists

Indicator C141 · Rate how effectively the two firms manage issues arising from agent work as one team rather than through two separate processes · Full statistical analysis ›

Separate issue trackers could leave the two firms with inconsistent statuses and unresolved problems, while favorable accounts described coordinated joint review. The mean rating was 3.7 and the median 4.0 (n = 799 respondents). Its mean of 3.73 was the highest of the 20 indicators of co-management (n = 799 respondents). Respondents on canceled engagements rated joint issue management lower than respondents on active engagements in any stage (mean 2.3, n = 302 respondents).

Figure 2.28. Issue Management Through Separate Lists: associations with the study measures

Scroll horizontally to view the complete figure.

Issue management through separate lists: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.61 [−0.70, −0.50] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.61 [0.47, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.85 [0.79, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.60] 749 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70428285099999999 | 1 |
| 95% confidence interval | 1 | -0.50234515430000004 | 1 |
| Reported association | 0 | -0.61323704999999995 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.46774219560000002 | 1 |
| 95% confidence interval | 1 | 0.72499563320000004 | 1 |
| Reported association | 0 | 0.6123273628 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.79459281770000001 | 1 |
| 95% confidence interval | 1 | 0.88687240540000001 | 1 |
| Reported association | 0 | 0.8469956649 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72201485809999999 | 1 |
| 95% confidence interval | 1 | -0.59782456240000004 | 1 |
| Reported association | 0 | -0.66386551469999999 | 1 |

Note: Survey question: “Rate how effectively the two firms manage issues arising from agent work as one team rather than through two separate processes”.

Distressed ratings: 391 of 799 answers (48.94%).

Group means and numeric-answer bases: Individual contributor (n = 616): 3.58. Engagement manager (n = 150): 4.04. Portfolio director (n = 31): 4.94. Build-Out (n = 162): 4.81. Transition/Cut-Over (n = 240): 4.57. Production (n = 95): 4.19. Canceled (n = 302): 2.33.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Co-management of issues as one team was associated with cancellation, customer satisfaction, the engagement indicator mean, and intent to leave, and no robustness check overturned those associations. Cancellation was less common where issues were managed jointly (r = −0.61, 95% CI −0.70 to −0.50, n = 150 engagements). Joint issue management was associated with higher customer satisfaction among the active engagements (r = 0.61, 95% CI 0.47 to 0.72, n = 94 engagements). It was associated with a higher engagement indicator mean, computed without the issue-management indicator (r = 0.85, 95% CI 0.79 to 0.89, n = 150 engagements). Respondents who rated joint issue management higher reported lower intent to leave (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.60, n = 749 respondents). Distressed ratings came from 391 of the 799 answers (48.94%), and the engagement mean was distressed on 63 of the 150 engagements.

Issue-management comments numbered 54, all supplied in the joint-management field. 39 were negative. Twenty-six described separate trackers that were not reconciled, sometimes with inconsistent statuses or an additional vendor-maintained list. Positive accounts described effective joint calls or a shared list. Unlike the prospective risks in section 2.1, these comments concern problems already requiring action.

> “we and the bank keep two separate issue logs, and neither is complete. Ours records what the agent got wrong, theirs records what reached their desk, and the overlap is maybe half. What that means in practice is this: when we sit down every fortnight we spend the first half hour agreeing what happened, before anyone can decide anything.” -Engagement Manager, KPO, Credit, underwriting and suitability

> “when an inquiry lands with the wrong relationship team, nobody on their side takes it, our lead takes it, thus it is our fault by default. Accordingly I stopped raising them. Two processes, and me in the middle.” -Individual Contributor, CXM, Client servicing and communications

> “In March an agent put hold on around forty accounts wrongly. Our side found it in an hour, but their servicing team was running its own investigation separately and we only joined up next day... same issue, two diffrent calls.” -Individual Contributor, BPO, Payments, cards and disputes

> “Weekly call with their service manager, Tuesday. Issues get logged both sides, one in their tool one in ours, so same ticket sits in two places.” -Individual Contributor, ITO, Technology engineering and operations

> “My marks in this section are high because of one thing, the joint triage call on thursday actually works, and their middle office lead reads every classification we flag. It is a person, not a process.” -Engagement Manager, KPO, Corporate and back-office functions

More negative issue-management sentiment was associated with higher intent to leave. The qualitative distinction is between the existence of records and coordination of resolution. A shared list does not by itself establish timely action or accountability.

### 2.3 Resolution Speed Against the Spread of Agent Error

Indicator C142 · Rate how well the speed of joint issue resolution matches the speed at which agent-caused problems can spread · Full statistical analysis ›

Written accounts described agent processing continuing while joint issue resolution and escalation remained incomplete. The mean rating was 3.7 and the median 4.0 (n = 809 respondents). Respondents on canceled engagements rated the speed of resolution lower than respondents on active engagements in any stage (mean 2.3, n = 306 respondents).

Figure 2.29. Resolution Speed Against the Spread of Agent Error: associations with the study measures

Scroll horizontally to view the complete figure.

Resolution speed against the spread of agent error: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.51] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.60 [0.46, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.84 [0.79, 0.88] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.64 [−0.71, −0.57] 758 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70617370229999998 | 1 |
| 95% confidence interval | 1 | -0.50515230889999996 | 1 |
| Reported association | 0 | -0.6155787466 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.45694626129999999 | 1 |
| 95% confidence interval | 1 | 0.71841659040000005 | 1 |
| Reported association | 0 | 0.60367219660000004 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.78687128829999997 | 1 |
| 95% confidence interval | 1 | 0.88239245550000001 | 1 |
| Reported association | 0 | 0.84106957319999998 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70742191799999998 | 1 |
| 95% confidence interval | 1 | -0.57286799440000002 | 1 |
| Reported association | 0 | -0.64496654880000004 | 1 |

Note: Survey question: “Rate how well the speed of joint issue resolution matches the speed at which agent-caused problems can spread”.

Distressed ratings: 400 of 809 answers (49.44%).

Group means and numeric-answer bases: Individual contributor (n = 628): 3.53. Engagement manager (n = 150): 3.85. Portfolio director (n = 30): 5.17. Build-Out (n = 168): 4.68. Transition/Cut-Over (n = 237): 4.58. Production (n = 98): 4.00. Canceled (n = 306): 2.26.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Each of the four core study measures was associated with the speed at which the two firms resolved issues together, and all applicable robustness checks upheld the associations. Faster resolution was associated with less cancellation (r = −0.62, 95% CI −0.71 to −0.51, n = 150 engagements). Among active engagements, customer satisfaction was higher where resolution was rated faster (r = 0.60, 95% CI 0.46 to 0.72, n = 94 engagements). The engagement indicator mean, computed without the resolution-speed indicator, was higher as well (r = 0.84, 95% CI 0.79 to 0.88, n = 150 engagements). Intent to leave was lower among respondents who rated resolution faster (r = −0.64, cluster bootstrap 95% CI −0.71 to −0.57, n = 758 respondents). Distressed ratings came from 400 of the 809 answers (49.44%), and the engagement mean was distressed on 61 of the 150 engagements.

Few written responses addressed the speed of resolution, 18, all in comment on co-management, and 16 of the 18 were negative. Most described an agent that kept acting while the two firms resolved the problem. One described 50 responses sent during a three-day escalation, and another 20 more new hires moved through a misfiring step before anyone saw the alert. A third described 100 more lines processed while the firms discussed a wrong supplier record.

> “When an onboarding journey misfires, the agent has already moved twenty more new hires through the same step before anyone reads the alert. What we need is a standing call, not a weekly one, and one named individual at the bank who can halt a journey without waiting for the Thursday forum.” -Engagement Manager, CXM, Corporate and back-office functions

> “when an agent-generated validation output turns out wrong, we spend the first day of the escalation deciding whose problem it is rather than fixing it. That part has not improved in nine months.” -Engagement Manager, KPO, Credit, underwriting and suitability

> “Practical change is who can stop the run. when matching goes wrong on a broker statement i must log it our side and again their side, two tickets, two waiting... by then breaks are already double. one named person per shift with right to pause the Agent, and one issue list instead of two.” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “It is better than first month, now we have one call together on Thursday and issue list is shared. Still if the policy Bot gives wrong answer on leave rules it stays like that till weekly call, nobody there can pause it” -Individual Contributor, CXM, Corporate and back-office functions

No significant association was detected in the written-sentiment analysis, which rests on only 18 responses and cannot establish the absence of a relationship. The accounts illustrate continued processing during unresolved incidents and difficulties reaching personnel authorized to suspend the agent.

### 2.4 Authority to Stop a Misbehaving Agent

Indicator C143 · Rate how quickly the two firms together can stop an agent that is behaving badly in live operations · Full statistical analysis ›

Respondents described uncertainty about who possessed both the authority and the technical means to stop a misbehaving agent. The mean rating was 3.6 and the median 3.0 (n = 804 respondents). Respondents on canceled engagements rated the ability to stop an agent lower than respondents on active engagements in any stage (mean 2.2, n = 312 respondents).

Figure 2.30. Authority to Stop a Misbehaving Agent: associations with the study measures

Scroll horizontally to view the complete figure.

Authority to stop a misbehaving agent: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.63 [−0.72, −0.52] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.60 [0.45, 0.71] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.86 [0.81, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.67 [−0.73, −0.61] 753 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71650355249999997 | 1 |
| 95% confidence interval | 1 | -0.5205634482 | 1 |
| Reported association | 0 | -0.62839990040000004 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.45029029059999998 | 1 |
| 95% confidence interval | 1 | 0.71433699669999995 | 1 |
| Reported association | 0 | 0.59831877320000004 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.80539757050000005 | 1 |
| 95% confidence interval | 1 | 0.89311221399999996 | 1 |
| Reported association | 0 | 0.85526625199999995 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72523330239999995 | 1 |
| 95% confidence interval | 1 | -0.60669445320000004 | 1 |
| Reported association | 0 | -0.67096701010000004 | 1 |

Note: Survey question: “Rate how quickly the two firms together can stop an agent that is behaving badly in live operations”.

Distressed ratings: 406 of 804 answers (50.50%).

Group means and numeric-answer bases: Individual contributor (n = 621): 3.49. Engagement manager (n = 150): 3.93. Portfolio director (n = 31): 4.71. Build-Out (n = 162): 4.57. Transition/Cut-Over (n = 236): 4.63. Production (n = 94): 4.12. Canceled (n = 312): 2.22.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Every association of the joint ability to stop a misbehaving agent with the four core study measures held, in each of the applicable robustness checks. Engagements that could stop an agent more quickly had a lower cancellation rate (r = −0.63, 95% CI −0.72 to −0.52, n = 150 engagements). Customer satisfaction among their active engagements was higher (r = 0.60, 95% CI 0.45 to 0.71, n = 94 engagements). The ability to stop an agent was closely associated with the engagement indicator mean, computed without the stopping indicator (r = 0.86, 95% CI 0.81 to 0.89, n = 150 engagements). Respondents who rated it higher reported lower intent to leave (r = −0.67, cluster bootstrap 95% CI −0.73 to −0.61, n = 753 respondents). Distressed ratings came from 406 of the 804 answers (50.50%), and the engagement mean was distressed on 62 of the 150 engagements.

Comments on suspension authority numbered 73, from 71 respondents on 51 engagements. 59 were negative. Nearly half described uncertainty over who could stop the agent or a perceived inability of either firm to do so. Some reported continued processing for hours or days. Several proposed assigning a named person with both authority and the technical means to suspend operation.

> “The agent started auto closing help desk tickets that were still open on the floor side, about forty of them in an hour, because someone had changed a rule on a Friday and nobody told us. So i go to the bank's IT lead and ask him to switch it off. He says he cannot, it is our tool. i ask our own side and they say the change has to go through the joint forum, and the forum sits every second week … In the end one of my analysts stopped it from the console himself and we wrote it up afterwards, and to this day nobody has told us whether that was allowed.” -Engagement Manager, CXM, Corporate and back-office functions

> “We spotted it fast and the client's credit head was on the call within the hour, but actually switching it off took three people and nearly two hours... the shared risk log worked, the stop button did not really exist.” -Engagement Manager, KPO, Credit, underwriting and suitability

> “One thing... nobody can switch it off. Last month the model start flagging every counterparty in one sector, hundreds of alerts in a morning. I ask my team lead who stops it and he does not know either. It runs the full cycle. Then we sit and clear the alert list one by one after.” -Individual Contributor, KPO, Credit, underwriting and suitability

> “In week six the generator produced a joining letter with the wrong notice period. Our QA caught it before any new hire saw it. But then we spent two days deciding who was permitted to switch the thing off, us or their HR team.” -Engagement Manager, CXM, Corporate and back-office functions

> “Escalation has improved, we can halt an agent inside the hour now and both sides accept it without argument.” -Portfolio Director, KPO, Regulatory, audit, tax and legal

More negative sentiment about suspension capability was associated with higher intent to leave. Favorable accounts distinguished rapid suspension from later disputes about remediation. Negative accounts described unavailable authorization or technical access and subsequent manual correction. The distinction separates the power to stop processing from responsibility for its consequences.

> The problem is nobody can decide anything. And I mean nobody, not my lead, not the person above him, not the people on their side who were meant to sign off the escalation matrix. So the fix is not complicated, you name one person over there with authority to say yes on a policy answer that is not in the handbook, and you give that person a number we can actually reach. That is it. That is the whole of it. Right now an edge case comes in, a leave entitlement question the handbook does not cover, and it goes into a queue nobody empties, which is why I end up writing the answer myself and hoping. And when it is wrong the ticket carries my name, not theirs. Second thing, and this one is smaller, somebody has to be allowed to switch the agent off for one queue without calling a meeting. We had three days where it was handing out the old parental leave policy and everybody knew, everybody, and it kept going out because turning it off was not anyone's decision to make. Three days. I do not think the people above understand that when nothing gets decided the work does not stop, it just gets done by whoever is sitting there. Which was us. Furthermore this was all raised in week two.” INDIVIDUAL CONTRIBUTOR · CXM, Corporate and back-office functions

## 3 Joint Change Control

Respondents described approval delays, updates discovered after implementation, and changes in behavior between formal releases as separate limits on joint change control. Joint change control is assessed through approval responsiveness, advance visibility of updates, and governance of behavioral changes between formal releases. The last construct is termed drift in the survey. A behavioral change does not by itself show autonomous self-modification: it may reflect inputs, provider updates, configuration, memory, or other causes not identified by the questionnaire.

The written accounts distinguish delayed approval, inadequate notice, and unmonitored behavioral variation. These are related control problems but require different responses: authority to approve, communication before implementation, and detection of changes outside the scheduled release process.

> “Here there is no such forum. Changes to the agent behaviour happen and I find out from my own team, not from the client and not from our side either.” -Engagement Manager, CXM, Financial crime and client onboarding

> “these questions assume both sides agree what change even is, we discuss about a change for weeks then agent behaviour move on its own in between” -Individual Contributor, ITO, Technology engineering and operations

Some respondents described informal approvals through messages or conversations. Whether these practices left the formal record incomplete requires comparison with actual change records. The survey did not perform that comparison.

### 3.1 Joint Change Control Outpaced by Agent Behavior

Indicator C144 · Rate how well the joint change control process keeps pace with the speed of agent behavior changes · Full statistical analysis ›

Joint change approvals were reported to operate more slowly than the changes required during agentic delivery. The mean rating was 3.7 and the median 3.5 (n = 798 respondents). Respondents on canceled engagements rated joint change control lower than respondents on active engagements in any stage (mean 2.2, n = 303 respondents).

Figure 2.31. Joint Change Control Outpaced by Agent Behavior: associations with the study measures

Scroll horizontally to view the complete figure.

Joint change control outpaced by agent behavior: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.63 [−0.71, −0.52] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.60 [0.45, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.60] 752 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71493063000000001 | 1 |
| 95% confidence interval | 1 | -0.51820853060000005 | 1 |
| Reported association | 0 | -0.62644453359999996 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.45294622289999997 | 1 |
| 95% confidence interval | 1 | 0.715967032 | 1 |
| Reported association | 0 | 0.60045653580000002 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.80239251249999999 | 1 |
| 95% confidence interval | 1 | 0.89138015709999996 | 1 |
| Reported association | 0 | 0.85296855319999998 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71950937989999997 | 1 |
| 95% confidence interval | 1 | -0.60227871320000004 | 1 |
| Reported association | 0 | -0.66439086739999997 | 1 |

Note: Survey question: “Rate how well the joint change control process keeps pace with the speed of agent behavior changes”.

Distressed ratings: 399 of 798 answers (50.00%).

Group means and numeric-answer bases: Individual contributor (n = 616): 3.50. Engagement manager (n = 150): 4.11. Portfolio director (n = 31): 5.03. Build-Out (n = 164): 4.66. Transition/Cut-Over (n = 236): 4.67. Production (n = 95): 4.08. Canceled (n = 303): 2.25.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Joint change control, as rated against the pace of agent change, was associated with all four core study measures, and each association held under the applicable robustness checks. Engagements whose change control kept better pace had a lower cancellation rate (r = −0.63, 95% CI −0.71 to −0.52, n = 150 engagements). Their customer satisfaction among active engagements was higher (r = 0.60, 95% CI 0.45 to 0.72, n = 94 engagements). Change control that kept pace was associated with a higher engagement indicator mean, computed without the change-control indicator (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated it higher reported lower intent to leave (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.60, n = 752 respondents). After Holm correction, ratings of joint change control also differed by process family. Distressed ratings came from 399 of the 798 answers (50.00%), and the engagement mean was distressed on 65 of the 150 engagements.

Joint-change-control comments numbered 71, from 70 respondents on 53 engagements. 57 were negative. Respondents described weekly, every-two-week, or monthly approval cycles that did not match the frequency of required changes. Some reported delays of days or weeks and changes agreed outside the formal process.

> “In March they changed the tolerance on the price check and we found out when the breaks tripled next morning. Nobody told us … The problem is the override queue, one firm waits for the other and the break sits four days. Accordingly nobody is accountable and everybody is polite.” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “Change requests reach me directly from the deal desk, in email, sometimes in a hallway call, and they are not raised as change orders, so the sprint absorbs them and the burndown reads as though we estimated badly … There is no shared record of what changed and when it changed.” -Engagement Manager, ITO, Technology engineering and operations

> “Practically, the reviewer on their side approves our rule changes in a batch every Thursday and cannot possibly read them all. We would need two named owners per rule set, one there and one here, and the authority to push a low-risk config change without waiting for the Thursday meeting” -Engagement Manager, ITO, Regulatory, audit, tax and legal

> “Ran change board with them again last week, same outcome. What must change is simple, both sides need to see rule updates before they take effect, not in following month's minutes. When their team altered retention period for claims files, our classifier kept filing to old rule for nearly three weeks, and nobody told us.” -Engagement Manager, BPO, Corporate and back-office functions

> “A tuning change to how alerts are ranked took eleven days to sign off on both sides” -Individual Contributor, ITO, Technology engineering and operations

More negative written sentiment about joint change control was associated with higher cancellation. The accounts identify approval latency and informal authorization as possible control weaknesses. They do not establish that the timing of meetings caused engagement outcomes.

### 3.2 Agent Updates Disclosed to the Other Firm After the Fact

Indicator C145 · Rate how well agent updates made by one firm become visible to the other firm before they take effect · Full statistical analysis ›

Respondents described discovering the other firm's agent updates after implementation rather than receiving advance visibility. The mean rating was 3.6 and the median 4.0 (n = 803 respondents). Respondents on canceled engagements rated the visibility of updates lower than respondents on active engagements in any stage (mean 2.2, n = 306 respondents).

Figure 2.32. Agent Updates Disclosed to the Other Firm After the Fact: associations with the study measures

Scroll horizontally to view the complete figure.

Agent updates disclosed to the other firm after the fact: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.51] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.60 [0.45, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.84 [0.79, 0.88] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.59] 756 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71203832410000001 | 1 |
| 95% confidence interval | 1 | -0.51388605580000002 | 1 |
| Reported association | 0 | -0.62285188979999995 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.45354696010000001 | 1 |
| 95% confidence interval | 1 | 0.71633532740000005 | 1 |
| Reported association | 0 | 0.60093977750000005 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.78522317340000003 | 1 |
| 95% confidence interval | 1 | 0.88143398839999998 | 1 |
| Reported association | 0 | 0.83980299680000003 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72108834759999996 | 1 |
| 95% confidence interval | 1 | -0.59454040370000005 | 1 |
| Reported association | 0 | -0.66224211769999997 | 1 |

Note: Survey question: “Rate how well agent updates made by one firm become visible to the other firm before they take effect”.

Distressed ratings: 394 of 803 answers (49.07%).

Group means and numeric-answer bases: Individual contributor (n = 621): 3.50. Engagement manager (n = 150): 3.89. Portfolio director (n = 30): 4.77. Build-Out (n = 166): 4.62. Transition/Cut-Over (n = 237): 4.56. Production (n = 94): 4.03. Canceled (n = 306): 2.24.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

All four core study measures were associated with the visibility of each firm's agent updates to the other before they took effect, and none of the applicable robustness checks overturned an association. Cancellation was less common where updates were visible in advance (r = −0.62, 95% CI −0.71 to −0.51, n = 150 engagements). Advance visibility of updates was associated with higher customer satisfaction among the active engagements (r = 0.60, 95% CI 0.45 to 0.72, n = 94 engagements). The engagement indicator mean, computed without the visibility indicator, was higher where updates were visible (r = 0.84, 95% CI 0.79 to 0.88, n = 150 engagements). Intent to leave was lower among respondents who saw updates in advance (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.59, n = 756 respondents). Distressed ratings came from 394 of the 803 answers (49.07%), and the engagement mean was distressed on 66 of the 150 engagements.

The written responses addressing the visibility of updates numbered 48, from 48 respondents on 44 engagements, and 38 of the 48 were negative. Most said that a change to an agent made by one firm reached the other only after it had taken effect. It was discovered from the output, from a failure, or at the next weekly meeting. Several respondents said the vendor's side did the same.

> “Somebody on their side changes what the assistant is allowed to answer and we find out from the tickets, not from a note. Last month it started closing password resets on its own and nobody on my team knew that was turned on. And afterwards we are asked why the numbers moved.” -Individual Contributor, CXM, Corporate and back-office functions

> “Every time someone on my team changes a prompt or a mapping we write it in The study with the date and who did it... And their engineers do the same thing on their side, only we never see theirs and they never see ours … So then we make a rule between ourselves. Before you push anything you ping the other three people on the team group. Not the official channel, the group chat. That is the whole change control now.” -Individual Contributor, ITO, Technology engineering and operations

> “Main thing is we find out the bot wording changed by hearing it come out on a live call, nobody tells us before, accordingly my team loses the first hour of shift working out what is different today” -Individual Contributor, CXM, Payments, cards and disputes

> “Last sprint we update the prompt template for the code review agent on friday, the bank QA lead only comes to know on monday when his report looks different. Nothing broke but he was not happy about it. After that we start putting every change in the shared tracker, still it is only us doing this, not them” -Individual Contributor, ITO, Technology engineering and operations

> “For any change agent make to the build script, my team put note in group chat and hold it till thursday release call... not official process but it stop surprises on their side.” -Individual Contributor, ITO, Technology engineering and operations

More negative update-visibility sentiment was associated with higher cancellation and intent to leave. Respondents described discovering changes after implementation. One proposed advance notice during the daily call. The study did not evaluate that proposal's effectiveness or implementation burden.

### 3.3 Governance of Behavioral Drift Between Releases

Indicator C146 · Rate how well the two firms jointly govern changes agents make to their own behavior between formal releases · Full statistical analysis ›

Changes in agent behavior between formal releases were reported to expose gaps in monitoring and ownership. Its wording described changes agents made to their own behavior, but the responses do not identify the technical cause of those changes. The mean was 3.7 and the median 4.0 (n = 806 respondents). Canceled engagements had a mean of 2.2 (n = 301), lower than each active stage.

Figure 2.33. Governance of Behavioral Drift Between Releases: associations with the study measures

Scroll horizontally to view the complete figure.

Governance of behavioral drift between releases: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.51] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.59 [0.44, 0.71] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.83 [0.78, 0.88] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.67 [−0.72, −0.60] 756 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70692951420000005 | 1 |
| 95% confidence interval | 1 | -0.50627557710000004 | 1 |
| Reported association | 0 | -0.6165152183 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.436183602 | 1 |
| 95% confidence interval | 1 | 0.70563095440000001 | 1 |
| Reported association | 0 | 0.58692885250000004 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.7762501571 | 1 |
| 95% confidence interval | 1 | 0.87620176049999998 | 1 |
| Reported association | 0 | 0.83289679439999997 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72210542330000005 | 1 |
| 95% confidence interval | 1 | -0.59585146580000004 | 1 |
| Reported association | 0 | -0.66506723310000004 | 1 |

Note: Survey question: “Rate how well the two firms jointly govern changes agents make to their own behavior between formal releases”.

Distressed ratings: 388 of 806 answers (48.14%).

Group means and numeric-answer bases: Individual contributor (n = 624): 3.55. Engagement manager (n = 150): 3.85. Portfolio director (n = 30): 4.97. Build-Out (n = 169): 4.76. Transition/Cut-Over (n = 239): 4.59. Production (n = 97): 4.04. Canceled (n = 301): 2.19.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

The joint governance of drift in agent behavior between releases was associated with cancellation, customer satisfaction, the engagement indicator mean, and intent to leave, and the associations held through every robustness check. Engagements that governed drift better had a lower cancellation rate (r = −0.62, 95% CI −0.71 to −0.51, n = 150 engagements). Among active engagements, governance of drift was associated with higher customer satisfaction (r = 0.59, 95% CI 0.44 to 0.71, n = 94 engagements). The engagement indicator mean, computed without the drift indicator, was higher where drift was governed better (r = 0.83, 95% CI 0.78 to 0.88, n = 150 engagements). Respondents who rated the governance of drift higher reported lower intent to leave (r = −0.67, cluster bootstrap 95% CI −0.72 to −0.60, n = 756 respondents). Distressed ratings came from 388 of the 806 answers (48.14%), and the engagement mean was distressed on 62 of the 150 engagements.

Fourteen written responses addressed behavioral drift, all in the joint-management field. 12 were negative. They described monitoring and ownership gaps for changes outside formal releases. The accounts do not establish whether those changes arose from autonomous modification, provider updates, configuration, or other causes.

> “the matching behaviour change on its own between releases and nobody on either side is watching for that... formal changes are handled fine, that part works, we get told and we test it. But when the match rate drift on the trading flow cash items there is no forum for it, my lead tell their ops lead informally and it stay there.” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “The checkpoints sit in the right spots. What nobody owns is when the bot quietly starts answering a ticket type diffrently than it did last month” -Individual Contributor, ITO, Technology engineering and operations

> “Tuesday joint call has helped, issues that used to sit for weeks now close in two days. Change control is the weak part still, agent updates itself between releases and neither side looks at it until a pipeline falls over” -Individual Contributor, ITO, Technology engineering and operations

> “Bot changed how it sorted leave cases and nobody told us... my team took blame for two weeks of backlog” -Individual Contributor, BPO, Corporate and back-office functions

> “What number cannot show is how quick both leads pick up the phone. Only gap is when the agent changes its own behaviour between releases, that we notice late” -Individual Contributor, BPO, Corporate and back-office functions

The written-sentiment tests did not detect significant associations on this 14-response base. Precision is limited. Respondents described discovering behavioral changes through altered output or failures, suggesting a need to distinguish release authorization from ongoing monitoring.

> The joint quality review scores accuracy of the shortlist. It does not score the interviews we had to rebook. Model behavior changes between releases are noted in our log only, the client's recruiting ops team is informed at the monthly meeting.” ENGAGEMENT MANAGER · CXM, Corporate and back-office functions

## 4 Human Review and the Joint Quality Assurance Loop

Reported human review weaknesses concerned where checks occurred, whether reviewers had sufficient capacity, and whether approval reflected substantive scrutiny. Five indicators describe how the two firms checked the agents' output. They cover where human review points were placed, whether review capacity kept up with the volume of output, and how much scrutiny output received at those points. The indicators also cover how reliably the joint quality loop caught agent errors before live operation, and whether the firms' quality measures captured the errors agents actually make. Each had about half of its answers in the distressed range, from 377 of 787 answers (47.90%) for the quality loop to 405 of 792 (51.14%) for the calibration of quality measures. Across the five, no robustness check overturned an association with the core study measures.

The review indicators distinguish placement, capacity, substantive scrutiny, detection procedures, and the adequacy of quality measures. Their common concern is whether approval corresponds to an effective check. Detailed counts and examples are retained under each function below.

> “Marked the review one lower for a reason. Most days volume is fine but Mondays it backs up alot and two of us are checking messages for the whole floor. Everything else about how the two sides work together has been fine.” -Individual Contributor, CXM, Client servicing and communications

The accounts describe review occurring after output had been released, approval without examination, and checks that missed plausible but incorrect results. These problems are not exclusive to agents. Their relevance here is whether controls addressed the outputs and failure modes of the deployed systems.

### 4.1 Placement of Human Review Points Across the Two Firms

Indicator C147 · Rate how well human review points are placed across the two firms at the moments where agent errors would matter most · Full statistical analysis ›

Some reported human review points occurred after release or relied on the producer to review the output, limiting the opportunity for independent intervention before an error reached its recipient. The mean rating was 3.6 and the median 4.0 (n = 797 respondents). Respondents on canceled engagements rated the placement of review lower than respondents on active engagements in any stage (mean 2.2, n = 303 respondents).

Figure 2.34. Placement of Human Review Points Across the Two Firms: associations with the study measures

Scroll horizontally to view the complete figure.

Placement of human review points across the two firms: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.51] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.60 [0.46, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.84 [0.79, 0.88] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.60] 747 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70868112169999997 | 1 |
| 95% confidence interval | 1 | -0.50888139089999995 | 1 |
| Reported association | 0 | -0.61868649190000002 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.4572004811 | 1 |
| 95% confidence interval | 1 | 0.71857205170000005 | 1 |
| Reported association | 0 | 0.60387640460000003 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.78901019679999995 | 1 |
| 95% confidence interval | 1 | 0.8836351643 | 1 |
| Reported association | 0 | 0.84271243880000002 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71712521490000003 | 1 |
| 95% confidence interval | 1 | -0.59588634289999998 | 1 |
| Reported association | 0 | -0.66175241979999999 | 1 |

Note: Survey question: “Rate how well human review points are placed across the two firms at the moments where agent errors would matter most”.

Distressed ratings: 393 of 797 answers (49.31%).

Group means and numeric-answer bases: Individual contributor (n = 614): 3.52. Engagement manager (n = 150): 3.90. Portfolio director (n = 31): 4.74. Build-Out (n = 168): 4.60. Transition/Cut-Over (n = 229): 4.66. Production (n = 97): 4.01. Canceled (n = 303): 2.23.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

The placement of human review points was associated with cancellation, customer satisfaction, the engagement indicator mean, and intent to leave, and the applicable robustness checks upheld each association. Cancellation was less common on engagements whose review points were rated well placed (r = −0.62, 95% CI −0.71 to −0.51, n = 150 engagements). Customer satisfaction among active engagements was higher where review points were rated well placed (r = 0.60, 95% CI 0.46 to 0.72, n = 94 engagements). The engagement indicator mean, computed without the placement indicator, was higher too (r = 0.84, 95% CI 0.79 to 0.88, n = 150 engagements). Respondents who rated the placement of review higher reported lower intent to leave (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.60, n = 747 respondents). Distressed ratings came from 393 of the 797 answers (49.31%), and the engagement mean was distressed on 65 of the 150 engagements.

Review-placement comments numbered 105, from 103 respondents on 75 engagements. 75 were negative. Accounts described checks occurring after release and, in some cases, output reviewed only by its producer. Other respondents considered placement appropriate but identified capacity or scrutiny problems, addressed in the following subsections.

> “Last week the agent matched a cash item onto the wrong value date and it went out with the pack, nobody had looked at it. For my manual clears there is a checker, it has always been that way, but whatever the agent matches goes straight through.” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “Agent does gross to net and then it just goes, there is no point in the run where somebody on their side puts a name against it. On our side my Lead checks what he can, but he has four people and one cycle is eleven thousand records, so he checks sample and that is it.” -Individual Contributor, BPO, Corporate and back-office functions

> “The checker on our side signs off two hundred fee lines a day and the approval on their side is a same-day formality, so a wrong tier rate can pass both and only surface when an advisor complains a month later.” -Engagement Manager, BPO, Payments, cards and disputes

> “We keep a two person rota for reading the drafted notes each morning before the desk opens, this is not written in the governance pack anywhere, we make it in week three ourselves.” -Engagement Manager, KPO, Corporate and back-office functions

More negative sentiment about review placement was associated with higher intent to leave. A correctly located review point can still lack capacity or substantive scrutiny. Conversely, a thorough check performed after release may not prevent the initial error from reaching its recipient.

### 4.2 Human Review Capacity Against Agent Output Volume

Indicator C148 · Rate how well joint human review capacity keeps up with the volume of output the agents produce · Full statistical analysis ›

Respondents described human review capacity that did not keep pace with agent output volume and additional checking assigned to analysts. The mean rating was 3.7 and the median 4.0 (n = 805 respondents). Respondents on canceled engagements rated review capacity lower than respondents on active engagements in any stage (mean 2.2, n = 304 respondents).

Figure 2.35. Human Review Capacity Against Agent Output Volume: associations with the study measures

Scroll horizontally to view the complete figure.

Human review capacity against agent output volume: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.52] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.59 [0.43, 0.70] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.82 [0.76, 0.87] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.67 [−0.72, −0.60] 755 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.7132418827 | 1 |
| 95% confidence interval | 1 | -0.51568352080000002 | 1 |
| Reported association | 0 | -0.6243464192 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.43393865500000001 | 1 |
| 95% confidence interval | 1 | 0.70423792529999996 | 1 |
| Reported association | 0 | 0.58511073359999999 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.76198958269999995 | 1 |
| 95% confidence interval | 1 | 0.86783755839999999 | 1 |
| Reported association | 0 | 0.82188453139999995 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72253461149999998 | 1 |
| 95% confidence interval | 1 | -0.60037143969999995 | 1 |
| Reported association | 0 | -0.66544026359999997 | 1 |

Note: Survey question: “Rate how well joint human review capacity keeps up with the volume of output the agents produce”.

Distressed ratings: 381 of 805 answers (47.33%).

Group means and numeric-answer bases: Individual contributor (n = 624): 3.54. Engagement manager (n = 150): 3.91. Portfolio director (n = 29): 5.17. Build-Out (n = 166): 4.73. Transition/Cut-Over (n = 238): 4.63. Production (n = 97): 4.01. Canceled (n = 304): 2.25.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Joint review capacity against the volume of agent output showed an association with each of the four core study measures, upheld in all applicable robustness checks. Engagements whose review capacity kept up had a lower cancellation rate (r = −0.62, 95% CI −0.71 to −0.52, n = 150 engagements). Customer satisfaction among their active engagements was higher (r = 0.59, 95% CI 0.43 to 0.70, n = 94 engagements). Review capacity was associated with a higher engagement indicator mean, computed without the capacity indicator (r = 0.82, 95% CI 0.76 to 0.87, n = 150 engagements). Intent to leave was lower among respondents who rated review capacity higher (r = −0.67, cluster bootstrap 95% CI −0.72 to −0.60, n = 755 respondents). Distressed ratings came from 381 of the 805 answers (47.33%), and the engagement mean was distressed on 65 of the 150 engagements.

Review-capacity comments numbered 145, from 135 respondents on 89 engagements. 134 were negative (92.41%). More than half came from the question about challenges reducing job satisfaction. Respondents described output volumes exceeding available review capacity and additional checking assigned to analysts. These accounts concern perceived capacity rather than a measured workload-to-staffing ratio.

> “the release notes and the config diffs the agent puts together land in a review queue and the review queue is me and one other person, and we are also the ones doing the actual releases. so we read the ones that touch card auth and we tick the rest, because there is no time.” -Individual Contributor, ITO, Payments, cards and disputes

> “Same for review capacity, there is no joint review, there is me.” -Individual Contributor, ITO, Client servicing and communications

> “review is mostly quick approve, i will be honest... when queue is long the checker just click through. but this is not because anyone is careless, the volume of claimant messages is very high and both side are only small team.” -Individual Contributor, CXM, Client servicing and communications

> “The validation queue backs up at month-end. Two weeks into the month, and we still haven't finished last month's QA.” -Individual Contributor, KPO, Corporate and back-office functions

> “Human review became the bottleneck everyone blamed but nobody fixed” -Individual Contributor, KPO, Regulatory, audit, tax and legal

More negative review-capacity sentiment was associated with higher intent to leave. Some reporting-alteration accounts described dashboards that omitted review backlogs (Part 5). Capacity constraints and reporting practices should be verified separately: one does not establish the other.

### 4.3 Routine Approval of Agent Output at Human Review Points

Indicator C149 · Rate how much genuine scrutiny agent outputs receive at human review points rather than routine approval · Full statistical analysis ›

Routine approval without substantive scrutiny was reported at human review points, particularly where review capacity did not match output volume. The mean rating was 3.6 and the median 4.0 (n = 794 respondents). Ratings were highest among active engagements in Build-Out (mean 4.7, n = 160 respondents) and lowest among canceled engagements (mean 2.2, n = 300 respondents). Individual contributors, the role whose output the review is meant to check, gave the lowest ratings of the roles (mean 3.5, n = 612 respondents).

Figure 2.36. Routine Approval of Agent Output at Human Review Points: associations with the study measures

Scroll horizontally to view the complete figure.

Routine approval of agent output at human review points: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.64 [−0.73, −0.53] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.62 [0.48, 0.73] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.69 [−0.74, −0.63] 747 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72550166989999998 | 1 |
| 95% confidence interval | 1 | -0.53409262499999999 | 1 |
| Reported association | 0 | -0.63960728590000004 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.48351272360000003 | 1 |
| 95% confidence interval | 1 | 0.73452255200000005 | 1 |
| Reported association | 0 | 0.62490870669999998 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.80384894240000004 | 1 |
| 95% confidence interval | 1 | 0.89221993960000001 | 1 |
| Reported association | 0 | 0.85408239959999999 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.74473405540000004 | 1 |
| 95% confidence interval | 1 | -0.63177350070000005 | 1 |
| Reported association | 0 | -0.69171085399999999 | 1 |

Note: Survey question: “Rate how much genuine scrutiny agent outputs receive at human review points rather than routine approval”.

Distressed ratings: 393 of 794 answers (49.50%).

Group means and numeric-answer bases: Individual contributor (n = 612): 3.50. Engagement manager (n = 150): 3.93. Portfolio director (n = 30): 4.97. Build-Out (n = 160): 4.73. Transition/Cut-Over (n = 236): 4.61. Production (n = 98): 4.10. Canceled (n = 300): 2.16.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

The scrutiny of agent output at human review points was associated with every core study measure, and every association held under each of the applicable robustness checks. Engagements whose output received more scrutiny had a lower cancellation rate (r = −0.64, 95% CI −0.73 to −0.53, n = 150 engagements). Customer satisfaction among active engagements was higher where output received more scrutiny (r = 0.62, 95% CI 0.48 to 0.73, n = 94 engagements). The rating was most closely associated with the engagement indicator mean, computed without the indicator (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated scrutiny higher reported lower intent to leave (r = −0.69, cluster bootstrap 95% CI −0.74 to −0.63, n = 747 respondents). Distressed ratings came from 393 of the 794 answers (49.50%), and the engagement mean was distressed on 64 of the 150 engagements.

The written responses addressing scrutiny numbered 66, from 63 respondents on 50 engagements, and 59 of the 66 (89.39%) were negative. The account given in 36 of the 50 responses written in answer to the indicator's own question was a review point that approved agent output without reading it. Respondents attributed it chiefly to review capacity that did not match the volume of output. Respondents who wrote rated scrutiny lower than respondents who did not, at a mean of 2.6 (n = 62 respondents) against 3.7 (n = 732 respondents).

Figure 2.37. Reported approval of agent output without substantive scrutiny

Scroll horizontally to view the complete figure.

Respondents described approval of agent output without reading it

Reported practice

Control implication

Evidence boundary

In 36 of 50 answers to the indicator’s own question, respondents described a review point that approved agent output without reading it.

Approval without substantive examination could create a misleading impression that review occurred.

The study did not inspect audit trails. This is a control implication of reported practice.

Note: The indicator drew 66 written answers from 63 respondents on 50 engagements. Fifty answers addressed the indicator’s own question. These bases differ.

Responses from 63 respondents on 50 engagements. 59 of the 66 (89.39%) were negative. A score of −10 marks the part of the engagement described at or near its worst.

> “Every release goes through a review step and someone ticks it, but nobody reads what the agent changed in the pipeline, it is a tick and nothing more. I have watched approvals come back in under a minute for a change that touched twenty files.” -Individual Contributor, ITO, Technology engineering and operations

> “We set a target of reviewing a sample of guided filings each day and we are not hitting it. In practice the reviewer opens the transcript, checks the loss date and the policy number match, and signs it. That is not scrutiny, that is a tick. Nobody has time for more than that at current volume, and volume is going up when the pilot widens in the fall.” -Engagement Manager, CXM, Client servicing and communications

> “Approvals come back same day mostly, but they are ticked through, nobody really reads the claimant message before it goes... so my team reads it twice ourselves instead.” -Individual Contributor, CXM, Client servicing and communications

> “We send exception pack for sign off on tuesday. Their approver return it in nine minutes with no comment, 300 lines. Same nine minutes every week, so we stop expecting real check and just fix it ourself” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “Last month the reviewer approve one batch of alerts in two minutes. Nobody can read 300 in two minutes.” -Individual Contributor, ITO, Regulatory, audit, tax and legal

> “Last month the agent shut a whole run of alerts overnight, because threshold was changed on their side. Review is one click, my lead approve forty at a time. Nobody catch it for three days” -Individual Contributor, ITO, Regulatory, audit, tax and legal

Written sentiment about scrutiny was not significantly associated with the study measures. Approval without substantive examination could create a misleading impression that review occurred. The study did not inspect audit trails, so this is a control implication of the reported practice rather than a verified finding about institutional records.

### 4.4 Reliability of Joint Quality Assurance

Indicator C150 · Rate how reliably the joint quality assurance loop catches agent errors before they reach live operations · Full statistical analysis ›

Reported weaknesses in joint quality assurance concerned the coverage and timing of error detection before live operation. The mean rating was 3.7 and the median 4.0 (n = 787 respondents). On canceled engagements, the quality loop was rated lower than on active engagements in any stage (mean 2.3, n = 297 respondents).

Figure 2.38. Reliability of Joint Quality Assurance: associations with the study measures

Scroll horizontally to view the complete figure.

Reliability of joint quality assurance: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.61 [−0.70, −0.50] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.60 [0.45, 0.71] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.85 [0.79, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.59] 742 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70329322230000002 | 1 |
| 95% confidence interval | 1 | -0.50087764840000004 | 1 |
| Reported association | 0 | -0.61201209670000001 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.44939206409999999 | 1 |
| 95% confidence interval | 1 | 0.71378507570000005 | 1 |
| Reported association | 0 | 0.59759531259999998 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.79451186139999996 | 1 |
| 95% confidence interval | 1 | 0.88682552520000002 | 1 |
| Reported association | 0 | 0.84693360029999998 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.7220033414 | 1 |
| 95% confidence interval | 1 | -0.59284782820000004 | 1 |
| Reported association | 0 | -0.66120721699999996 | 1 |

Note: Survey question: “Rate how reliably the joint quality assurance loop catches agent errors before they reach live operations”.

Distressed ratings: 377 of 787 answers (47.90%).

Group means and numeric-answer bases: Individual contributor (n = 605): 3.55. Engagement manager (n = 150): 3.93. Portfolio director (n = 30): 4.80. Build-Out (n = 158): 4.70. Transition/Cut-Over (n = 235): 4.60. Production (n = 97): 4.07. Canceled (n = 297): 2.29.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

All four core study measures were associated with the reliability of the joint quality assurance loop, and none of the applicable robustness checks overturned an association. A more reliable quality loop was associated with less cancellation (r = −0.61, 95% CI −0.70 to −0.50, n = 150 engagements). Among active engagements, customer satisfaction was higher where the loop was rated reliable (r = 0.60, 95% CI 0.45 to 0.71, n = 94 engagements). The engagement indicator mean, computed without the quality-loop indicator, was higher as well (r = 0.85, 95% CI 0.79 to 0.89, n = 150 engagements). Respondents who rated the loop reliable reported lower intent to leave (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.59, n = 742 respondents). Distressed ratings came from 377 of the 787 answers (47.90%), and the engagement mean was distressed on 60 of the 150 engagements.

Quality-assurance comments numbered 29, mostly in the joint-management field. 22 were negative. Respondents described checks limited to typing or formatting, checks occurring after release, and separate team procedures outside the joint process. These accounts concern the coverage and timing of detection.

> “Last month the assistant told an employee our parental leave was 12 weeks off the old policy pdf. Logs showed where it pulled it from. Nothing caught it before it went out.” -Individual Contributor, CXM, Corporate and back-office functions

> “the checks we run were built for human mistakes, transposed digits and such, they would never catch the way the agent gets a break wrong.” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “i felt embarrassed. tool generate welcome pack with wrong leave policy, it reach new joiner, two days nobody decide who fix it, in end my team do.” -Individual Contributor, CXM, Corporate and back-office functions

> “QA method for agent decisions never got built before launch” -Individual Contributor, BPO, Financial crime and client onboarding

More negative quality-assurance sentiment was associated with higher cancellation and intent to leave. Reported examples included outdated source material and incorrect classification. These errors can occur in human or automated work. The relevant issue is whether the checks used on the engagement detected them.

### 4.5 Coverage of Agent Errors in Quality Measures

Indicator C151 · Rate how well the quality measures both firms use capture the kinds of errors agents actually make · Full statistical analysis ›

Some written accounts described quality measures that missed the types of error respondents encountered in agent output. The mean rating was 3.6 and the median 3.0 (n = 792 respondents). Respondents on canceled engagements rated the quality measures lower than respondents on active engagements in any stage (mean 2.2, n = 300 respondents).

Figure 2.39. Coverage of Agent Errors in Quality Measures: associations with the study measures

Scroll horizontally to view the complete figure.

Coverage of agent errors in quality measures: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.61 [−0.71, −0.50] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.63 [0.49, 0.74] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.60] 743 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.7051806692 | 1 |
| 95% confidence interval | 1 | -0.50367752310000002 | 1 |
| Reported association | 0 | -0.61434874009999996 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.49439685709999998 | 1 |
| 95% confidence interval | 1 | 0.74104040810000005 | 1 |
| Reported association | 0 | 0.63354917899999996 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.81352209389999997 | 1 |
| 95% confidence interval | 1 | 0.89778204539999995 | 1 |
| Reported association | 0 | 0.86146854520000005 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71557605199999996 | 1 |
| 95% confidence interval | 1 | -0.59668764460000001 | 1 |
| Reported association | 0 | -0.66021884050000001 | 1 |

Note: Survey question: “Rate how well the quality measures both firms use capture the kinds of errors agents actually make”.

Distressed ratings: 405 of 792 answers (51.14%).

Group means and numeric-answer bases: Individual contributor (n = 609): 3.48. Engagement manager (n = 150): 3.79. Portfolio director (n = 31): 4.97. Build-Out (n = 165): 4.54. Transition/Cut-Over (n = 231): 4.59. Production (n = 96): 3.99. Canceled (n = 300): 2.21.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Every association of the calibration of quality measures to agent errors with the four core study measures held, in each of the applicable robustness checks. Engagements whose quality measures captured agent errors had a lower cancellation rate (r = −0.61, 95% CI −0.71 to −0.50, n = 150 engagements). Calibrated quality measures were associated with higher customer satisfaction among the active engagements (r = 0.63, 95% CI 0.49 to 0.74, n = 94 engagements). The engagement indicator mean, computed without the calibration indicator, was closely associated with them (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Intent to leave was lower among respondents who rated the calibration higher (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.60, n = 743 respondents). Distressed ratings came from 405 of the 792 answers (51.14%), and the engagement mean was distressed on 66 of the 150 engagements.

Quality-measure comments numbered 14. 11 were negative. Seven described inherited measures that did not capture the errors respondents encountered in agent output. Examples included an outdated policy response, totals that concealed incorrect allocations, and a mapping error that generated no exception.

> “What neither firm measures is the kind of mistake the agent actually makes, which is a funding position that is right in total and wrong per merchant. Our quality sampling counts breaks cleared, consequently it scores that as a pass.” -Engagement Manager, BPO, Trade, settlement and reconciliation

> “The checking is not real. Approver just sign off the vendor onboarding queue, they do not open the record. And the QA sheet do not have any line for the mistake the bot actually make” -Individual Contributor, CXM, Corporate and back-office functions

> “The quality checks we agreed cover accuracy of the policy answer only. They do not cover the answer being out of date, which is what actually goes wrong. That gap is open since we start.” -Engagement Manager, CXM, Corporate and back-office functions

> “On my last posting it is a claims process and the QA is just sampling ten files a week. Here they decide things much faster, which I like. But the checks we run still look for the mistakes a person makes... the agent does not make those. It makes a different kind.” -Individual Contributor, BPO, Payments, cards and disputes

> “Data-transformation error rates were reported only for errors that triggered a system exception, so the silent hallucinated mappings that ran without throwing an error never appeared in the official error rate at all.” -Individual Contributor, ITO, Technology engineering and operations

Written-sentiment tests reached significance for cancellation and intent to leave, but the 14-response base makes these exploratory results imprecise and potentially sensitive to individual observations. They should not be dismissed solely because the base is small, or treated as conclusive because a significance threshold was crossed. The substantive accounts identify gaps in what the quality measures captured.

## 5 The Shift From Managing People to Managing Agents

Joint governance was reported to provide insufficient cross-firm visibility or decision speed for work performed by agents. Four indicators describe whether the co-management of the engagement had been adapted to work performed by agents. They ask whether each firm could see what the other firm's agents had done and whether joint decisions kept pace with the agents. The remaining two ask whether the joint governance model fit work performed by agents and whether it was ready for the autonomy the respondent expected within two years. Each had about half of its answers in the distressed range, from 381 of 786 answers (48.47%) for the speed of joint decisions to 407 of 808 (50.37%) for readiness for future autonomy. Readiness was the lowest-rated indicator of the category. Each of the four was associated with every core study measure, and the applicable robustness checks left those associations intact.

The comments raise questions about whether governance arrangements accommodated automated operation. The distinct issues are cross-firm visibility, decision authority and timing, overall suitability, and preparedness for greater autonomy. Section 3 addresses change-control latency in detail. It is not repeated here as a separate mechanism.

> “the joint model is not failing, it is simply calibrated to a cadence that no longer exists. The governance forum meets monthly, which was entirely reasonable when a change meant a release … It is a design that predates the thing it governs, and no one on either side has yet been willing to say so out loud.” -Portfolio Director, CXM, Corporate and back-office functions

> “governance can't match agent speed” -Engagement Manager, ITO, Technology engineering and operations

Some respondents considered institutional and vendor personnel capable but constrained by inherited governance arrangements. That interpretation is part of their reported experience rather than an independently tested explanation.

### 5.1 Agent Actions Visible Only Inside Each Firm's Own Tools

Indicator C152 · Rate how fully each firm can see what the other firm’s agents actually did when questions arise · Full statistical analysis ›

Limited visibility into the other firm's agent actions left respondents describing manual reconstruction from separate statements, logs, or spreadsheets. The mean rating was 3.7 and the median 3.0 (n = 780 respondents). Visibility was weakest on canceled engagements, where the mean rating was 2.2 (n = 293 respondents), lower than on active engagements in any stage.

Figure 2.40. Agent Actions Visible Only Inside Each Firm's Own Tools: associations with the study measures

Scroll horizontally to view the complete figure.

Agent actions visible only inside each firm's own tools: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.62 [−0.71, −0.51] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.62 [0.48, 0.73] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.86 [0.81, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.60] 732 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.70786728840000002 | 1 |
| 95% confidence interval | 1 | -0.50767021999999995 | 1 |
| Reported association | 0 | -0.61767750200000004 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.48038813949999998 | 1 |
| 95% confidence interval | 1 | 0.73264282970000005 | 1 |
| Reported association | 0 | 0.62242180999999996 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.80712028739999997 | 1 |
| 95% confidence interval | 1 | 0.89410398290000004 | 1 |
| Reported association | 0 | 0.85658257559999995 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.7210309801 | 1 |
| 95% confidence interval | 1 | -0.59969585319999996 | 1 |
| Reported association | 0 | -0.66384259010000002 | 1 |

Note: Survey question: “Rate how fully each firm can see what the other firm’s agents actually did when questions arise”.

Distressed ratings: 392 of 780 answers (50.26%).

Group means and numeric-answer bases: Individual contributor (n = 598): 3.55. Engagement manager (n = 150): 3.89. Portfolio director (n = 30): 5.13. Build-Out (n = 165): 4.65. Transition/Cut-Over (n = 230): 4.74. Production (n = 92): 3.88. Canceled (n = 293): 2.25.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Cross-firm visibility was associated with a lower observed cancellation rate (r = −0.62, 95% CI −0.71 to −0.51, n = 150 engagements), higher active-engagement customer satisfaction (r = 0.62, 95% CI 0.48 to 0.73, n = 94), and a higher leave-one-out indicator mean (r = 0.86, 95% CI 0.81 to 0.89, n = 150). Respondents rating visibility more favorably reported lower intent to leave (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.60, n = 732 respondents). These associations passed the applicable sensitivity checks and concern realized cancellation rather than rated future likelihood. Distressed ratings came from 392 of the 780 answers (50.26%), and the engagement mean was distressed on 65 of the 150 engagements.

The written responses addressing visibility numbered 66, from 64 respondents on 49 engagements, and 62 of the 66 were negative. Respondents described each firm's agents as opaque to the other. When a question arose, an account of what an agent had done was rebuilt by hand, from statements, logs, or a private spreadsheet.

> “Their release manager and I keep a running note — what our agent touched, what their pipeline did, timestamps from both sides — because neither firm's tooling shows the other's actions and we got tired of arguing from memory on the Monday call. We update it by hand each evening … Formal risk log gets written afterwards, from that note.” -Engagement Manager, ITO, Technology engineering and operations

> “A release went out and half the config was not what we signed off, we asked their side what their automation had changed and nobody could show us, they only have the final state not the steps. We spend two days rebuilding it from our own notes.” -Individual Contributor, ITO, Technology engineering and operations

> “The vendor buries agent decision logs inside a proprietary trace format our audit team cannot query directly, so every review request becomes a manual export they control the timing and scope of.” -Individual Contributor, ITO, Technology engineering and operations

> “When a requirement comes back wrong we cannot see what their side's tool did with the file, only what came out, so the first hour goes to reconstructing it. … Decision speed itself is acceptable. The visibility is not.” -Engagement Manager, BPO, Credit, underwriting and suitability

> “Observability logs get trimmed to the final agent decision only, so the intermediate steps that would show where a judgment call went wrong never make it into the audit trail the vendor shares with us.” -Individual Contributor, CXM, Corporate and back-office functions

Cancellation was more common where the written responses on visibility were more negative. Respondents who wrote more negatively about visibility reported higher intent to leave. In answer to the question on how the vendor altered its reports, respondents described agent logs trimmed, aggregated, or held in a format the financial institution's auditors could not query (Part 5).

### 5.2 Joint Decision Speed Mismatched to Agent Speed

Indicator C153 · Rate how closely the speed of joint decision making matches the speed at which the agents act · Full statistical analysis ›

The speed of joint decisions was reported to lag the pace at which agents acted and operational changes required authorization. The mean rating was 3.7 and the median 4.0 (n = 786 respondents). The speed of joint decisions received a mean rating of 2.2 on canceled engagements (n = 294 respondents), lower than on active engagements in any stage.

Figure 2.41. Joint Decision Speed Mismatched to Agent Speed: associations with the study measures

Scroll horizontally to view the complete figure.

Joint decision speed mismatched to agent speed: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.64 [−0.72, −0.53] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.62 [0.47, 0.73] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.86 [0.81, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.67 [−0.73, −0.61] 738 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72449028670000004 | 1 |
| 95% confidence interval | 1 | -0.53256704740000005 | 1 |
| Reported association | 0 | -0.63834575810000005 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.47389537770000001 | 1 |
| 95% confidence interval | 1 | 0.72872449640000003 | 1 |
| Reported association | 0 | 0.61724494949999997 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.80692534370000002 | 1 |
| 95% confidence interval | 1 | 0.89399179650000005 | 1 |
| Reported association | 0 | 0.85643365189999998 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72842097269999995 | 1 |
| 95% confidence interval | 1 | -0.60664780679999997 | 1 |
| Reported association | 0 | -0.67201728090000001 | 1 |

Note: Survey question: “Rate how closely the speed of joint decision making matches the speed at which the agents act”.

Distressed ratings: 381 of 786 answers (48.47%).

Group means and numeric-answer bases: Individual contributor (n = 604): 3.57. Engagement manager (n = 150): 3.83. Portfolio director (n = 30): 5.30. Build-Out (n = 163): 4.67. Transition/Cut-Over (n = 235): 4.65. Production (n = 94): 4.23. Canceled (n = 294): 2.21.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Where joint decisions kept pace with the agents, all four core study measures were more favorable, and each association survived the applicable robustness checks. Cancellation was less common where joint decisions kept pace (r = −0.64, 95% CI −0.72 to −0.53, n = 150 engagements). Active engagements with faster joint decisions reported higher customer satisfaction (r = 0.62, 95% CI 0.47 to 0.73, n = 94 engagements). The engagement indicator mean, computed without the decision-speed indicator, moved closely with the rating (r = 0.86, 95% CI 0.81 to 0.89, n = 150 engagements). Respondents who saw joint decisions keep pace with the agents reported lower intent to leave (r = −0.67, cluster bootstrap 95% CI −0.73 to −0.61, n = 738 respondents). Distressed ratings came from 381 of the 786 answers (48.47%), and the engagement mean was distressed on 65 of the 150 engagements.

Decision-speed comments numbered 14, all in the joint-management field. 13 were negative. Respondents described operational changes requiring decisions faster than the joint process supplied them. Some considered the governance design otherwise appropriate but its authorization schedule unsuitable for the work.

> “the agent does in an hour what then waits nine days for two people to agree on. That gap is the whole problem” -Individual Contributor, BPO, Credit, underwriting and suitability

> “Decisions here take longer than the work does. That is the whole of it really … Here it goes into a shared log that both firms look at and neither one closes. I have watched an item sit in there for a month. Nobody is being difficult about it. Their side thinks we have it, we think they have it.” -Individual Contributor, ITO, Technology engineering and operations

> “We escalated a flagged exposure last month and it took nearly two weeks to get a joint call on it... by then the account had already moved two grades. That is the thing I want the reader to understand. The agents work in minutes and the two firms still discuss about it for a fortnight.” -Individual Contributor, KPO, Credit, underwriting and suitability

> “We meet Tuesday to agree access changes, the agent works through joiner queue every hour. So a bad role assignment can sit live for six days before anyone talks about it. And nobody has said what we do when that is two hundred accounts instead of two.” -Individual Contributor, ITO, Technology engineering and operations

> “it asks how quickly the two firms decide together, and for that there must be a decision somewhere, i have not seen one yet. we sent up the question of which flagged events are safe for the RMs to act on, that was month two, it is still sitting there and the project is closed now.” -Individual Contributor, KPO, Client servicing and communications

> “When the agent picks wrong supplier record, nobody is switching it off. I raise, then two firms discuss about it and by then hundred more lines are done” -Individual Contributor, BPO, Corporate and back-office functions

No significant association was detected in the written-sentiment tests on this 14-response base. The accounts identify delays in authorization. They do not estimate the effect of faster decisions on outcomes. Related change-approval practices are described in section 3.1.

### 5.3 Joint Governance Model Designed for Work Performed by People

Indicator C154 · Rate how well today’s joint governance model fits work performed by agents rather than work performed by people · Full statistical analysis ›

Respondents described joint governance arrangements retained from work performed by people without corresponding changes in decision authority for agentic delivery. The mean rating was 3.6 and the median 4.0 (n = 803 respondents). On canceled engagements the mean rating for the fit of the model was 2.2 (n = 301 respondents), lower than respondents gave on active engagements in any stage.

Figure 2.42. Joint Governance Model Designed for Work Performed by People: associations with the study measures

Scroll horizontally to view the complete figure.

Joint governance model designed for work performed by people: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.64 [−0.72, −0.53] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.60 [0.45, 0.72] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.60] 758 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.72225030879999996 | 1 |
| 95% confidence interval | 1 | -0.52919268210000003 | 1 |
| Reported association | 0 | -0.63555341460000003 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.45188963589999998 | 1 |
| 95% confidence interval | 1 | 0.71531891219999999 | 1 |
| Reported association | 0 | 0.59960634030000004 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.80180055220000002 | 1 |
| 95% confidence interval | 1 | 0.89103865609999999 | 1 |
| Reported association | 0 | 0.85251570300000001 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.7205418093 | 1 |
| 95% confidence interval | 1 | -0.59745936399999999 | 1 |
| Reported association | 0 | -0.66273400609999999 | 1 |

Note: Survey question: “Rate how well today’s joint governance model fits work performed by agents rather than work performed by people”.

Distressed ratings: 395 of 803 answers (49.19%).

Group means and numeric-answer bases: Individual contributor (n = 621): 3.47. Engagement manager (n = 150): 3.93. Portfolio director (n = 30): 5.23. Build-Out (n = 164): 4.62. Transition/Cut-Over (n = 236): 4.60. Production (n = 102): 4.05. Canceled (n = 301): 2.19.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Every association of the fit between the joint governance model and agent work with the four core study measures held, in each of the applicable robustness checks. A model fitted to agent work was associated with a lower cancellation rate (r = −0.64, 95% CI −0.72 to −0.53, n = 150 engagements). Among active engagements, customer satisfaction was higher where the model fit agent work (r = 0.60, 95% CI 0.45 to 0.72, n = 94 engagements). The fit of the model was closely associated with the engagement indicator mean computed without it (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Intent to leave was lower among respondents who judged the model fit for agents (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.60, n = 758 respondents). Distressed ratings came from 395 of the 803 answers (49.19%), and the engagement mean was distressed on 67 of the 150 engagements.

Few written responses addressed the fit of the governance model, 10, nearly all in comment on co-management, and 9 of the 10 were negative. The most common account was a governance model drawn up for people performing the tasks by hand and left in place after agents took over much of it. Respondents described decisions that neither firm's people in the room could authorize and agent changes that both firms learned of afterward.

> “The governance we have was written for a team of people doing checks by hand, and now most of the checking is not done by hand at all. So the shape of it is wrong from the start.” -Individual Contributor, BPO, Corporate and back-office functions

> “There was a call on a Thursday in February where their reconciliation manager and I sat for forty minutes, agreed exactly what should happen with the unmatched cash items, and both of us knew that neither of us could authorise it … What we were handed was a governance model written for forty people doing manual matching, and then agents were put underneath it and nobody rewrote a line of it.” -Engagement Manager, BPO, Trade, settlement and reconciliation

> “when it was people doing this, my lead could just decide. now there is a agent doing the matching and nobody want to be the one who says it is allowed to post something. so it waits. i dont think anyone was lazy. it is just nobody was given the authority.” -Individual Contributor, BPO, Trade, settlement and reconciliation

> “The incident that sticks with me was a retraining push on the resolution model that went in on a Thursday and only came up in the following weeks steering call, by which point the desk had already seen a run of odd closures and worked around them. Nobody did anything wrong, procedurally, which is sort of the point. … What we dont have is a shared picture of what changed and who owns it when the agent gets something wrong” -Portfolio Director, ITO, Technology engineering and operations

No significant association was detected between governance-fit sentiment and the study measures on the 10-response base. The small sample limits precision. These comments concern the allocation of authority within the joint model, complementing the separate measures of decision speed and update visibility.

### 5.4 Readiness of the Joint Model for Expected Agent Autonomy

Indicator C155 · Rate how ready the joint management model is for the level of agent autonomy you expect in this engagement within two years · Full statistical analysis ›

Readiness for the agent autonomy expected within two years received the lowest mean rating among the co-management indicators. The mean rating was 3.60, the lowest of the 20 indicators of co-management, and the median 3.0 (n = 808 respondents). Readiness was rated lowest on canceled engagements (mean 2.2, n = 306 respondents), lower than on active engagements in any stage.

Figure 2.43. Readiness of the Joint Model for Expected Agent Autonomy: associations with the study measures

Scroll horizontally to view the complete figure.

Readiness of the joint model for expected agent autonomy: higher ratings accompanied more favorable outcomes

Scroll horizontally to view the complete comparison.

| Comparison | Correlation r | Estimate and analysis base |
| --- | --- | --- |
| Cancellation | | r = −0.63 [−0.72, −0.52] 150 engagements p < 0.001 |
| Customer satisfaction | | r = 0.63 [0.49, 0.74] 94 active engagements p < 0.001 |
| Indicator mean | | r = 0.87 [0.82, 0.90] 150 engagements p < 0.001 |
| Intent to leave | | r = −0.66 [−0.72, −0.60] 761 respondents p < 0.001 |
| | −1 0 +1 | |

Exact comparison data

Cancellation

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71863423459999998 | 1 |
| 95% confidence interval | 1 | -0.52375818269999996 | 1 |
| Reported association | 0 | -0.6310504186 | 1 |

Customer satisfaction

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.4901055627 | 1 |
| 95% confidence interval | 1 | 0.73847614699999997 | 1 |
| Reported association | 0 | 0.63014663400000004 | 1 |

Indicator mean

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | 0.82448776739999996 | 1 |
| 95% confidence interval | 1 | 0.90405502169999996 | 1 |
| Reported association | 0 | 0.8698171616 | 1 |

Intent to leave

Exact chart data

| Series | Point index | X | Y |
| --- | --- | --- | --- |
| 95% confidence interval | 0 | -0.71910792300000004 | 1 |
| 95% confidence interval | 1 | -0.59887856380000004 | 1 |
| Reported association | 0 | -0.66146238059999996 | 1 |

Note: Survey question: “Rate how ready the joint management model is for the level of agent autonomy you expect in this engagement within two years”.

Distressed ratings: 407 of 808 answers (50.37%).

Group means and numeric-answer bases: Individual contributor (n = 626): 3.46. Engagement manager (n = 150): 3.91. Portfolio director (n = 31): 4.77. Build-Out (n = 165): 4.52. Transition/Cut-Over (n = 238): 4.59. Production (n = 99): 4.03. Canceled (n = 306): 2.19.

r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational.

The engagement indicator mean excludes the tested indicator.

Readiness of the joint model for the autonomy expected within two years was associated with all four core study measures, and each association held under the applicable robustness checks. Engagements whose model was rated ready had a lower cancellation rate (r = −0.63, 95% CI −0.72 to −0.52, n = 150 engagements). Customer satisfaction among active engagements was higher where readiness was rated higher (r = 0.63, 95% CI 0.49 to 0.74, n = 94 engagements). The engagement indicator mean computed without the readiness indicator was higher as well (r = 0.87, 95% CI 0.82 to 0.90, n = 150 engagements). Respondents who judged the model ready reported lower intent to leave (r = −0.66, cluster bootstrap 95% CI −0.72 to −0.60, n = 761 respondents). Distressed ratings came from 407 of the 808 answers (50.37%), and the engagement mean was distressed on 64 of the 150 engagements.

Only three written responses addressed readiness for future autonomy: two negative and one neutral. The negative accounts referred to existing control problems, including separate issue logs and a review point that detected an error after a letter had been sent. They provide examples, not a basis for estimating prevalence.

> “The review point sits at the end only. If the agent codes a dispute reason wrongly it is found after the letter is already sent, not before. For the autonomy expected in two years this model will not hold.” -Individual Contributor, BPO, Payments, cards and disputes

The ratings measure respondents' assessment of readiness for the autonomy they expected within two years. They are neither a forecast of that autonomy nor a test of future governance performance. The three comments illustrate how current experiences informed some respondents' judgments.

> Simple version, someone has to be allowed to say yes. Right now if the model needs a new discount rate assumption, my lead asks their analyst, their analyst asks someone else, and it comes back four days later as a maybe. What would fix it is a name against each assumption, somebody over there who owns it, and if that person is out then a backup name. Thats it. That alone would take a week out of every deal. Second thing, when the agent changes how it pulls comparables we find out by reading the output. There should be a note somewhere that says what changed and when, we dont have that, so every time the numbers move we spend the morning proving it wasnt us. I have done that maybe six times now. And the review points are fine where they are, thats not the problem, the problem is nobody knows who is on the hook if a number goes out wrong. My honest guess is it would land on us. Nobody has ever said that out loud, but thats how it feels, and if it was written down somewhere at least we would know. Would definately slow us down for a month and then be faster after.” INDIVIDUAL CONTRIBUTOR · KPO, Corporate and back-office functions · “

In my last posting the process was written down and we get proper training first. Here nothing is written, we learn from whoever sit next to us. Trainings are only generic slides, no fund accounting in it”

INDIVIDUAL CONTRIBUTOR

BPO, Trade, settlement and reconciliation

← Financial Institution ReadinessVendor Capability →


## Links to related study evidence

- [Part 2](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/)
- [intent to leave](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-4-c019/)
- [customer satisfaction](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-1-measure-customer-satisfaction/)
- [Part 3](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-3-governance-oversight-control/)
- [engagement indicator mean](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-1-measure-engagement-indicator-mean/)
- [Part 5](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-5-manual-alteration/)


## Statistical analyses referenced in this paper

- [Effectiveness of Co-Management Structures](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c136/)
- [Unsettled Boundary of Agent Autonomy](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c137/)
- [Unowned Handoffs in Shared Workflows](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c138/)
- [Allocation of Accountability Between the Firms](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c139/)
- [Separate Risk Records Maintained by the Two Firms](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c140/)
- [Issue Management Through Separate Lists](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c141/)
- [Resolution Speed Against the Spread of Agent Error](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c142/)
- [Authority to Stop a Misbehaving Agent](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c143/)
- [Joint Change Control Outpaced by Agent Behavior](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c144/)
- [Agent Updates Disclosed to the Other Firm After the Fact](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c145/)
- [Governance of Behavioral Drift Between Releases](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c146/)
- [Placement of Human Review Points Across the Two Firms](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c147/)
- [Human Review Capacity Against Agent Output Volume](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c148/)
- [Routine Approval of Agent Output at Human Review Points](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c149/)
- [Reliability of Joint Quality Assurance](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c150/)
- [Coverage of Agent Errors in Quality Measures](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c151/)
- [Agent Actions Visible Only Inside Each Firm's Own Tools](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c152/)
- [Joint Decision Speed Mismatched to Agent Speed](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c153/)
- [Joint Governance Model Designed for Work Performed by People](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c154/)
- [Readiness of the Joint Model for Expected Agent Autonomy](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-2-c155/)