Co-Management and Governance Across the Other Four Categories

Co-management ratings consistently connected shared operating arrangements with governance in the other four categories. The 20 indicators concern authority, review, change, evidence, and response across the financial institution and vendor. Their low-rating share was 49.26%, based on 7,866 of 15,968 ratings. The governance, oversight, and control score provides a common outcome for examining these responsibilities, while the adjusted analysis remains conditional on its unequal category composition.

All 20 co-management indicators were strongly associated with the score after the tested indicator was omitted, with engagement-level r = 0.90–0.93. Correlations with the other four categories were 0.75–0.80. Under the main adjustment, partial r ranged from 0.21 to 0.31. All 20 met nominal p < 0.05 and survived Holm adjustment when considered as one 20-test family. The inference changes with the family: only two of the 20 survive Holm adjustment across all 47 cross-category tests. Subgroup results reuse overlapping respondents and engagements and are not independent replications.

The adjusted comparison has an important asymmetry. All 20 co-management indicators are in the score and none is in the 53-item comparison mean. That mean instead contains 18 other vendor capability indicators, 13 readiness indicators, 12 AI technology indicators, and 10 third-party indicators. Removing an item's category from the score also leaves different numbers of items in the outcome. The results identify associations conditional on this construction. They do not establish that co-management uniquely causes weaknesses in the other categories.

Among respondents who rated a co-management indicator 1 to 3, 61.19% to 65.21% had a score of 3 or below, against 2.65% to 5.30% among those who rated it 4 to 7. None of the respondents who rated a co-management indicator 8 to 10 had a score of 3 or below. Table 3.8 sets out the 20 indicators against these measures.

What a number cannot carry is that the two governance structures never met. We had our review, they had theirs, and no forum existed where the two were reconciled and someone could say who owned a step when an agent had performed it.”

PORTFOLIO DIRECTOR · KPO, Corporate and back-office functions

1 Joint Governance and Decision Rights

Co-management, decision rights, step ownership, and accountability each had strong observed relationships with the wider score. Together they define who can decide, who must act, and who owns the response when agent work goes wrong. Their similar statistical patterns support a common concern about shared authority, while each indicator addresses a distinct responsibility boundary. The evidence does not establish that a formal assignment alone makes that authority effective.

“These questions assume somebody actually decides. From where I sit nothing is decided, it is only carried to next weeks call and then the one after. Cannot put number of 1 to 10 on a thing that never happens” -Individual Contributor, KPO, Corporate and back-office functions

1.1 Co-Management of the Engagement

Indicator C136 · Rate how effectively the customer and vendor jointly manage this engagement now that agents perform part of the work · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 3.6 · Full statistical analysis ›

Weak co-management was associated with lower scores beyond the tested indicator. Co-management needs to coordinate the resolution of the same operational problem across the two firms. Written accounts describe forums that existed but did not settle decisions, alongside accounts of more effective joint working. That contrast makes the operation of coordination, rather than the number of negative comments, the central finding.

Scores were 3 or below for 249 of 385 (64.68%) at indicator ratings 1–3. The corresponding counts were 12 of 371 (3.23%) at indicator ratings 4–7 and 0 of 33 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 339.21, p < 0.001, Cramér's V = 0.66.

Co-management was associated with the score at r = 0.91 (95% CI 0.88 to 0.93, n = 150 engagements) and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.21 (p = 0.010), the second-weakest adjusted association of the 20.

The association was observed within the canceled engagements at r = 0.78 and across the active stages at 0.80 to 0.84, and it held after Holm adjustment in 85 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.85 among engagement managers, and 0.87 among portfolio directors. Among individual contributors alone, 66.88% (206 of 308) of those who rated co-management 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.71 (n = 385), against 5.23 (n = 404) among the rest (cluster-robust p < 0.001).

Co-management drew 209 written answers from 199 respondents on 115 engagements, the most of the 20 indicators, and 94 of them came from canceled engagements. Of the 127 negative answers, 83 came from canceled and 44 from active engagements, and 28 (22.05%) described one of the 26 forms of a control bypassed or not working, most often problems raised with no response (17), change discovered only through failure (6), and raising problems seen as futile (5). The negative answers described joint forums that existed on paper and decided nothing. The answers that were not negative described working forums beside a control that still failed, most often the authority to let an agent act without review.

“I want to be fair, the score is low because of one thing, escalation. We have a weekly joint call and it is polite and nothing gets decided in it. When the agent started pulling market share numbers from the wrong vintage of a report, I raised it on that call, their side logged it in their tracker, we logged it in ours, and for three weeks each side thought the other one was fixing it. My own management told me to keep it at working level and not put it in the client summary. So there are two versions of the issue list and neither one is complete. That is what the 1s are, not the people.” -Engagement Manager, KPO, Corporate and back-office functions

“Where I felt genuine progress is in the joint forum itself. In the first month the two sides came to it with separate issue logs, and we spent most of the hour reconciling numbers rather than deciding anything; that is behind us now, there is one list, both parties own it, and the weekly escalation call rarely runs over. Where there has been no movement is the question of who may authorise an agent to close an alert without a human looking at it. That has been open since the design workshops, it has been discussed about at every steering committee, and each time it is deferred to the next one, largely because neither side wishes to be the party whose name sits against that decision if a hit is missed. Review capacity is the same story in a different form.” -Portfolio Director, KPO, Financial crime and client onboarding

Part 2 (Co-Management, section 1.1) reports the association of co-management with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 3.6) its association with manual alteration.

1.2 Decision Rights Over Agent Action Without Human Approval

Indicator C137 · Rate how clearly the two firms have settled who decides what agents may do without human approval · Full statistical analysis ›

Unclear authority for agent action was associated with a weaker reported control environment. Decision rights determine which actions require human approval and which the two firms permit an agent to take independently. Where that boundary remains unsettled, a formal management structure may leave operational authority ambiguous. The ratings establish an association with the score, while the study does not directly observe every authorization decision or unauthorized action.

The 406 respondents who rated decision rights 1 to 3 included 257 with a score of 3 or below. Among the 375 who rated them 4 to 7, 12 (3.20%) had a score of 3 or below, and none of the 27 who rated them 8 to 10 did (chi-square = 331.00, p < 0.001, Cramér's V = 0.64).

Decision rights were associated with the score at r = 0.91 (95% CI 0.88 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.77. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.25 (p = 0.002), the sixth-strongest of the 20.

The association was observed within the canceled engagements at r = 0.77 and across the active stages at 0.80 to 0.84, and after Holm adjustment it held in 85 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.85 among engagement managers, and 0.86 among portfolio directors. Among individual contributors alone, 65.24% (214 of 328) of those who rated decision rights 1 to 3 had a score of 3 or below. Respondents who rated them distressed recorded a score of 2.74 (n = 406), against 5.30 (n = 402) among the rest (cluster-robust p < 0.001).

Forty-eight respondents on 42 engagements wrote about the limit of agent autonomy, one answer each, and 34 of the answers were negative, 13 from canceled and 21 from active engagements. Five of the 34 described one of the 26 forms of a control bypassed or not working, three of them problems raised with no response. Respondents described reviewers who approved agent work on trust in the absence of a written limit, and edge cases that reached a manager every evening. One individual contributor on a reconciliation engagement set out the missing control in three parts, a named approver on each side, one shared list, and a written limit on what the agent could clear alone. An engagement manager on another reconciliation engagement described what the missing limit cost the team.

“i will use this box properly since nobody asked before. what would have to change is one name. one person on their side and one on ours who can say yes to a break over a certain amount, that is all. right now a break comes up, i put it in the shared tracker, their ops person says it needs their treasury, treasury says the custodian must confirm, custodian says ask the client. ten days gone. also by then the next cycle is on top of it and the same break is now three breaks. so second thing, the tracker itself. we have ours and they have theirs and the numbers dont agree, and so half of every call is spent agreeing which list is correct instead of clearing anything. one list. it doesnt need a system, a spreadsheet would do. third, and this is the real one, somebody has to write down what the agent is allowed to clear on its own. under five hundred, matched to the statement line, clear it. we never got that number. so everything comes to a human, and the humans are waiting on each other. i said this in month two, kindly nothing came of it. which is why i am not hopeful, but you asked what would change it and that is what” -Individual Contributor, BPO, Trade, settlement and reconciliation

“What this costs us is people. My analysts are checking machine output at nine at night, work a tool was supposed to remove, because neither side will put a name against what the agent may clear on its own. Two of my better ones have asked to move. I have nothing to offer them to stay.” -Engagement Manager, BPO, Trade, settlement and reconciliation

Part 2 (Co-Management, section 1.2) reports the association of decision rights with the four study measures.

1.3 Ownership of Each Workflow Step

Indicator C138 · Rate how clearly ownership of each workflow step is assigned when agents, vendor staff, and customer staff work in the same process · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 3.7 · Full statistical analysis ›

Unclear ownership of workflow steps was associated with low scores. This concerns the point at which agents, delivery team members, and financial institution staff share a process but responsibility for a particular step is not settled. The governance issue is the assignment of responsibility across the workflow, distinct from accountability after an error. Similar score relationships across these controls do not make those responsibilities interchangeable.

Of the 388 respondents who rated step ownership 1 to 3, 253 had a score of 3 or below. Among the 392 who rated it 4 to 7, 15 (3.83%) had a score of 3 or below, and none of the 28 who rated it 8 to 10 did (chi-square = 345.81, p < 0.001, Cramér's V = 0.65).

Step ownership was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.78. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.25 (p = 0.003).

The association was observed within the canceled engagements at r = 0.74, the joint weakest of the 20 together with genuine scrutiny at review points, and it ranged across the active-stage groups from 0.80 in Build-Out to 0.85 in Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.86 among engagement managers, and 0.82 among portfolio directors. Among individual contributors alone, 66.67% (212 of 318) of those who rated step ownership 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.68 (n = 388), against 5.22 (n = 420) among the rest (cluster-robust p < 0.001).

Step ownership drew 82 written answers from 81 respondents on 58 engagements, and 70 of them were negative, 39 from canceled and 31 from active engagements. Of the 70, 21 (30.00%) described one of the 26 forms of a control bypassed or not working, most often problems raised with no response (11), change discovered only through failure (6), and raising problems seen as futile (4). Respondents described errors that passed between the two firms with each assuming the other had checked, and decisions that waited for weeks while each firm assigned them to the other. On one canceled engagement the team spent an hour on a call establishing which firm owned an error and then corrected it itself.

“the position is like this. there is no single owner of the allocation steps. my team prepares the expense split, the tool proposes the allocation, our lead looks at it, then it goes across to the fund controller side, and nowhere is it written which of those is the decision. so in practice the last person to touch the file is the decision. second thing, the tool behaves differently after an update and there is no joint forum where that is tabled. we find out because the output looks different that morning. i mention it in the daily call, that is the whole of the process. third, nobody has told us what the tool is permitted to do without a person approving. so i assume nothing, and i open each line myself, do note this roughly doubles the time on close days. the review on their side is a sign off and not a review. they sign what we send, and i am not saying they are wrong for that, they cannot check what they cannot see from their end. and when a query does come back it is three weeks later and the person who did the work is on another fund by then. one more, when we change something on our side there is no way for them to see it before it takes effect, and the same in reverse. that is the picture from where i sit” -Individual Contributor, BPO, Trade, settlement and reconciliation

“honestly my team is losing patience. when something go wrong nobody say which side own it, so we sit in call for hour and then still we fix it ourself. two of my colleague already ask for transfer” -Individual Contributor, BPO, Trade, settlement and reconciliation

Part 2 (Co-Management, section 1.3) reports the association of step ownership with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 3.7) its association with manual alteration.

1.4 Accountability for Problems Arising From an Agent's Action

Indicator C139 · Rate how clearly accountability lands on one firm when an agent's action causes a problem in this engagement · Full statistical analysis ›

Accountability for agent error was strongly associated with the wider score. Its control function is to establish which firm owns the response when an agent's action causes a problem. This differs from assigning routine workflow steps: an engagement may have detailed task ownership and still leave remediation responsibility unresolved. The overall association and the low-score share within a rating band describe different aspects of that relationship.

The indicator was associated with the score at r = 0.93 (95% CI 0.90 to 0.95, n = 150 engagements), second among the 47 governance indicators to the readiness of the joint model for expected agent autonomy. Scores were 3 or below for 246 of 402 (61.19%) at indicator ratings 1–3. The corresponding counts were 12 of 364 (3.30%) at indicator ratings 4–7 and 0 of 33 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 309.30, p < 0.001, Cramér's V = 0.62.

The strong overall association and the conditional low-score share answer different questions. Accountability correlated with governance in the other four categories at r = 0.80. After adjustment for the comparison mean, partial r was 0.22 (nominal p = 0.007). These results support a model-qualified relationship with the wider control environment, without a tested ranking of accountability's importance against other controls.

The association was observed within the canceled engagements at r = 0.76 and across the active stages at 0.82 to 0.88, and after Holm adjustment it held in 86 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.88 among engagement managers, and 0.89 among portfolio directors. Among individual contributors alone, 62.58% (204 of 326) of those who rated accountability 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.76 (n = 402), against 5.35 (n = 397) among the rest (cluster-robust p < 0.001).

Accountability drew 51 written answers from 50 respondents on 37 engagements, and 45 were negative, 20 from canceled and 25 from active engagements. Of the 45, 11 (24.44%) described one of the 26 forms of a control bypassed or not working, most often unclear liability or regulatory accountability (5), change discovered only through failure (3), and problems raised with no response (2). Respondents described errors that passed through three parties, each of which had done what was asked. An engagement manager on a payments engagement where an agent could be stopped within minutes asked for one line, settled in advance, naming the firm that owned remediation.

“I want to spend the whole of this box on one point, because the rest of it is ordinary. Ownership of a fee correction is, I would say, still unassigned. The agent proposes the adjustment, our analyst validates it, the bank's billing supervisor approves it, and on paper that is three clean steps. In practice, when a correction is wrong, all three of those parties can point at the other two and each of them is correct. I have run four engagements in this space and this is the first where I could not tell you, in a sentence, who owns the number that reaches the client statement. We discuss about it at every steering call and it never lands.” -Portfolio Director, BPO, Payments, cards and disputes

“We can stop an agent in minutes, that part is solved and both sides know the number to call. What is not solved is who pays when it has already sent two hundred wrong merchant replies before we stopped it. i would want one line in the operating manual saying which firm owns remediation, decided in advance and not argued after. Also give us a shared change window, right now their release calendar and ours are separate documents.” -Engagement Manager, CXM, Payments, cards and disputes

Part 2 (Co-Management, section 1.4) reports the association of accountability with the four study measures.

2 Shared Risk and Issue Management

Shared risk and issue management were associated with the broader control environment. A single view of risk, one working issue process, timely resolution, and the ability to stop an agent address different parts of the same operational response. The findings concern how respondents assessed those functions, not simply whether each firm had a documented process. Their association supports examining coordination across the boundary between firms.

“Review capacity has improved, I will say that. We have the hands now to check what the agents throw out each morning. What has not moved is stopping. If an agent starts matching breaks wrongly nobody can tell me who on either side is permitted to switch it off… we would still be mailing at noon.” -Engagement Manager, BPO, Trade, settlement and reconciliation

2.1 A Single Shared View of Agent Risk

Indicator C140 · Rate how well the two firms maintain a single shared view of the risks that agent work introduces · Full statistical analysis ›

A weak shared view of agent risk was associated with low scores. Separate risk records can leave the firms evaluating different parts of the same exposure, even where each maintains a formal risk process. The indicator concerns whether those views are brought together for joint oversight. The written accounts provide limited illustration of that arrangement, while the principal evidence comes from the ratings and their association with the broader score.

Of the 383 respondents who rated the shared view of agent risk 1 to 3, 245 (63.97%) had a score of 3 or below. Among the 384 who rated it 4 to 7, 13 (3.39%) had a score of 3 or below, and none of the 26 who rated it 8 to 10 did (chi-square = 333.60, p < 0.001, Cramér's V = 0.65).

The shared view of risk was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.78. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.25 (p = 0.002).

The association ranged across the active-stage groups, from 0.79 in Build-Out to 0.86 in Transition/Cut-Over and 0.89 in Production, and held within the canceled engagements at r = 0.75. After Holm adjustment, it was significant in 85 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.89 among engagement managers, and 0.83 among portfolio directors. Among individual contributors alone, 64.76% (204 of 315) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.71 (n = 383), against 5.24 (n = 410) among the rest (cluster-robust p < 0.001).

The written base was thin. Twenty-six respondents on 24 engagements wrote about a shared view of agent risk, 19 of the answers were negative, and 4 of the 19 described one of the 26 forms of a control bypassed or not working. The answers described two risk logs that met once a month, an exposure that remained on one side for a full cycle, and the same issue raised in two meetings and resolved in neither.

“The 6 on the shared risk view is narrower than it looks. We keep our risk log, they keep theirs, and the two only meet at the monthly forum, so a latency exposure we raised in week one sat on our side alone until the next cycle came round. The rest of the joint model holds. Who may restart what without asking us was settled early and has not caused an argument since.” -Engagement Manager, ITO, Technology engineering and operations

“my 1s here look harsh, mostly it is one thing… nobody keep a single risk list, so same issue get raise in two different meetings and fixed in neither. the people are polite, the process just does not exist” -Individual Contributor, ITO, Payments, cards and disputes

Part 2 (Co-Management, section 2.1) reports the association of the shared view of risk with the four study measures.

2.2 Co-Management of Issues Arising From Agent Work

Indicator C141 · Rate how effectively the two firms manage issues arising from agent work as one team rather than through two separate processes · Full statistical analysis ›

Fragmented issue management was associated with a weaker reported control environment. The relevant distinction is whether the two firms manage an agent-related problem together or pass it between separate processes. Shared management affects who can see the issue, assign action, and confirm resolution. The reported associations appear within both active and canceled groups, without establishing a change in this relationship over an engagement's life.

Within canceled engagements, r was 0.80. Only the visibility of each firm's agent actions to the other stood higher, at 0.82. Across the active stages the association ran at 0.82 in Build-Out and Transition/Cut-Over and 0.89 in Production, and after Holm adjustment it held in 85 of the 87 segment levels tested.

Scores were 3 or below for 249 of 391 (63.68%) at indicator ratings 1–3. The corresponding counts were 12 of 374 (3.21%) at indicator ratings 4–7 and 0 of 34 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 335.06, p < 0.001, Cramér's V = 0.65. The indicator was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.78. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.26 (p = 0.001), the second-strongest adjusted association of the 20.

Within role the association stood at 0.87 among individual contributors and at 0.88 among both engagement managers and portfolio directors. Among individual contributors alone, 64.58% (206 of 319) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.71 (n = 391), against 5.30 (n = 408) among the rest (cluster-robust p < 0.001).

Fifty-four respondents on 50 engagements wrote about joint issue management, one answer each, and 39 of the answers were negative, 16 from canceled and 23 from active engagements. Of the 39, 11 (28.21%) described one of the 26 forms of a control bypassed or not working, most often problems raised with no response (5). Respondents described two issue logs that agreed on perhaps half of their entries, merged queues agreed and never built, and alerts closed manually that appeared in no count. An engagement manager described two issue logs in which neither firm could see the other's agent actions, so each dispute became an argument over whose log was right.

“Two issue logs, theirs and ours. Neither side can see what the other's agents did. Every dispute becomes a discussion about whose record is right.” -Engagement Manager, ITO, Technology engineering and operations

“we keep our own issue tracker and the bank compliance side keep theirs. When i log a false positive pattern it stays on our side, i have never once seen one cross over. And nobody has ever asked how many alerts we dispose by hand in a day, that number is written nowhere.” -Individual Contributor, ITO, Financial crime and client onboarding

Part 2 (Co-Management, section 2.2) reports the association of joint issue management with the four study measures.

2.3 Resolution Speed Against the Spread of Agent-Caused Problems

Indicator C142 · Rate how well the speed of joint issue resolution matches the speed at which agent-caused problems can spread · Full statistical analysis ›

Low ratings of joint resolution speed were associated with low scores. For governance, the important comparison is between the time required to resolve an issue and the speed at which an agent-related problem can spread. The study measures respondents' assessment of that fit rather than elapsed incident-resolution times. The result supports examining operational response capacity while leaving the sequence and consequences of individual incidents unmeasured.

Among respondents rating resolution speed at 4–7, 18 of 380 (4.74%) had a score of 3 or below. This was the second-highest share among the 20 co-management indicators, after human review capacity at 5.30%. Of the 400 respondents who rated resolution speed 1 to 3, 248 (62.00%) had a score of 3 or below, and none of the 29 who rated it 8 to 10 did (chi-square = 304.28, p < 0.001, Cramér's V = 0.61).

Resolution speed was associated with the score at r = 0.91 (95% CI 0.88 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.77. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.25 (p = 0.002).

The association was observed within the canceled engagements at r = 0.78 and across the active stages at 0.79 to 0.82, and after Holm adjustment it held in 86 of the 87 segment levels tested. Within role it stood at 0.86 among both individual contributors and engagement managers and at 0.79 among portfolio directors. Among individual contributors alone, 63.38% (206 of 325) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.73 (n = 400), against 5.26 (n = 409) among the rest (cluster-robust p < 0.001).

The written base was thin. Eighteen respondents on 17 engagements wrote about resolution speed, and 16 of the answers were negative, 7 from canceled and 9 from active engagements. Three of the 16 described problems raised with no response. On one canceled engagement, 11 days passed before the two firms established whose change board owned an error, while three monthly attribution packs went to the financial institution's investment committee. On an active engagement the joint review met weekly, so a bad run was four days old before it was discussed, and the team kept a private list of the agent's changes in its place.

“In February the agents began handling the interaction effect differently after a platform update on our side — nobody had asked for that. It took eleven days to establish whose change board owned the question, by which time three monthly attribution packs had gone to the client's investment committee. Neither firm could show the other what had actually been done in that window. We closed it by recalculating the quarter by hand.” -Portfolio Director, KPO, Trade, settlement and reconciliation

“the joint review is once a week and by then a bad run is four days old, so we run a private list of what the agent changed and revert back on chat when it looks off. that sheet is the real control, not the meeting” -Individual Contributor, ITO, Technology engineering and operations

Part 2 (Co-Management, section 2.3) reports the association of resolution speed with the four study measures.

2.4 Speed of Stopping a Misbehaving Agent in Live Operations

Indicator C143 · Rate how quickly the two firms together can stop an agent that is behaving badly in live operations · Full statistical analysis ›

Limited ability to stop a misbehaving agent was associated with low scores, including a positive cross-category relationship under the adjusted model. This indicator concerns immediate operational control: the firms must be able to interrupt agent activity when a problem is recognized. It is distinct from routine approval, decision speed, and later remediation. The association does not estimate the causal benefit of a particular stop mechanism.

The adjusted cross-category estimate was partial r = 0.26 (nominal p = 0.001). The speed of stopping was associated with governance in the other four categories at r = 0.79, and its adjusted association (p = 0.001) was the third-strongest of the 20. It was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements).

Scores were 3 or below for 261 of 406 (64.29%) at indicator ratings 1–3. The corresponding counts were 13 of 374 (3.48%) at indicator ratings 4–7 and 0 of 24 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 333.22, p < 0.001, Cramér's V = 0.64. Respondents who rated it distressed recorded a score of 2.70 (n = 406), against 5.28 (n = 398) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.78 and across the active stages at 0.81 to 0.87, and after Holm adjustment it held in 85 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.88 among engagement managers, and 0.82 among portfolio directors. Among individual contributors alone, 66.07% (220 of 333) of those who rated it 1 to 3 had a score of 3 or below.

The speed of stopping drew 73 written answers from 71 respondents on 51 engagements, and 59 were negative, 21 from canceled and 38 from active engagements. Of the 59, 13 (22.03%) described one of the 26 forms of a control bypassed or not working, most often problems raised with no response (8) and the absence of an escalation path outside working hours (4). Respondents described three different names given for the person who could switch an agent off, and overnight shifts without a person authorized to pause one.

“the thing the numbers don't show is nobody could actually switch the thing off. i asked who does that and got three different names” -Individual Contributor, CXM, Corporate and back-office functions

“problem is at night no one over there can authorise stopping the triage bot, i have to raise ticket and wait for morning shift. one named person each side per shift with authority to pause the same, that is all it needs. the alerts it auto closes are the risk, not the ones it leaves open.” -Individual Contributor, ITO, Technology engineering and operations

Part 2 (Co-Management, section 2.4) reports the association of the speed of stopping with the four study measures.

3 Joint Change Control

The three change-control indicators connect the pace and visibility of agent change with the wider score. They distinguish a joint process that can keep pace, notice that reaches the other firm before effect, and governance of behavior between formal releases. These are complementary control functions. A strong association for one does not establish that the others can be treated as substitutes or that their relative coefficients rank causal importance.

“What we do now is we keep a group chat and that is the real record. Anything that needs a decision goes there, somebody screenshots it, and that becomes the approval. Nobody signed anything. If you ask me who owns the rule change from last month I will show you a chat message from a Tuesday, that is all there is. On paper there is a joint forum every two weeks. Half the time it is cancelled, the other half they send somebody who takes notes and says he will come back to us. He does not come back. So we made our own list, three of us keep it, we tick items off and if nobody objects in two days we go ahead. That is not governance, I know that, but the work has to move and no one else will move it. The other thing is the review. It is me reviewing my teammate and him reviewing mine, we are both tired and we both want to go home, so the second look is not really a second look. I raise this, feedbacks go up, nothing come down. Then when something breaks the question is always why did nobody check, and I do not have a good answer for that anymore.” -Individual Contributor, ITO, Regulatory, audit, tax and legal

3.1 Joint Change Control Against the Speed of Agent Behavior Change

Indicator C144 · Rate how well the joint change control process keeps pace with the speed of agent behavior changes · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 3.8 · Full statistical analysis ›

Joint change-control ratings retained a positive cross-category association under the specified adjustment. The operational concern is whether the two firms can govern changes at the pace at which agent behavior changes, rather than relying on a formal process whose cycle is too slow for the work. The observed coefficient is among the largest positive estimates in this comparison. Its ordering does not establish that this control has the largest causal effect.

Change control was associated with governance in the other four categories at r = 0.79, and after that adjustment the association remained at partial r = 0.31 (p < 0.001). It was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements), and after Holm adjustment the association held in 87 of the 87 segment levels tested.

Scores were 3 or below for 249 of 399 (62.41%) at indicator ratings 1–3. The corresponding counts were 15 of 360 (4.17%) at indicator ratings 4–7 and 0 of 39 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 310.22, p < 0.001, Cramér's V = 0.62. Its 1 to 3 band share was among the lowest of the 20 co-management indicators, while its adjusted association was the highest. These comparisons answer different questions. The rating bands describe the distribution of low governance, oversight, and control scores. The adjusted association describes the relationship remaining after accounting for the comparison mean. Their rankings do not provide a common scale of practical importance.

The association was observed within the canceled engagements at r = 0.79, the third-strongest of the 20, and ranged across the active-stage groups from 0.79 in Build-Out to 0.87 in Production. Within role it stood at 0.87 among individual contributors, 0.89 among engagement managers, and 0.78 among portfolio directors. Among individual contributors alone, 63.41% (208 of 328) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.74 (n = 399), against 5.25 (n = 399) among the rest (cluster-robust p < 0.001).

Joint change control drew 71 written answers from 70 respondents on 53 engagements, and 57 were negative, 25 from canceled and 32 from active engagements. Of the 57, 14 (24.56%) described one of the 26 forms of a control bypassed or not working, most often change discovered only through failure (8) and problems raised with no response (5). Respondents described a change request that waited five weeks while the agent kept working on the old rule, and changes that waited for a joint decision and then had to be made in haste.

“In march we change the tone rule for one client segment. i put in a change request. It sit for five week. In that time the tool send out maybe forty letters on the old rule and nobody flag it. When the request finally come back approved, the rule is already changed by somebody else, a different way. Nobody tell me. i find out from another analyst. There is no one risk list, both side keep their own” -Individual Contributor, CXM, Client servicing and communications

“they never decide anything together. a mid-term change sit two weeks and then we are told to fix it fast” -Individual Contributor, BPO, Client servicing and communications

Part 2 (Co-Management, section 3.1) reports the association of joint change control with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 3.8) its association with manual alteration.

3.2 Visibility of Agent Updates to the Other Firm Before They Take Effect

Indicator C145 · Rate how well agent updates made by one firm become visible to the other firm before they take effect · Full statistical analysis ›

Poor advance visibility of agent updates was associated with low scores. The control concerns whether one firm's change becomes known to the other before it affects shared work. Notification after a change takes effect serves a different purpose from an opportunity to assess and prepare for it. The findings connect reported update visibility with the wider control environment without directly measuring every update or response.

Of the 394 respondents who rated the visibility of agent updates 1 to 3, 250 (63.45%) had a score of 3 or below. Among the 376 who rated it 4 to 7, 16 (4.26%) had a score of 3 or below, and none of the 33 who rated it 8 to 10 did (chi-square = 321.39, p < 0.001, Cramér's V = 0.63).

The visibility of updates was associated with the score at r = 0.91 (95% CI 0.88 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.77. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.25 (p = 0.002).

The association was observed within the canceled engagements at r = 0.78, the fifth-strongest of the 20, and across the active stages at 0.82 to 0.84. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.88 among engagement managers, and 0.80 among portfolio directors. Among individual contributors alone, 64.51% (209 of 324) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.72 (n = 394), against 5.23 (n = 409) among the rest (cluster-robust p < 0.001).

Forty-eight respondents on 44 engagements wrote about the visibility of updates, one answer each, and 38 of the answers were negative, 12 from canceled and 26 from active engagements. Of the 38, 17 (44.74%) described one of the 26 forms of a control bypassed or not working, the highest proportion among the co-management indicators with 30 or more answers, and 14 of the 17 described change discovered only through failure. Respondents learned of the other firm's updates from failed cases and from figures that moved. A portfolio director reported that decision rights had been written down while updates still went live before the financial institution's conduct team knew of them.

“one point only. when they change something on the agent side we find out from the output, not from anybody telling us” -Individual Contributor, BPO, Payments, cards and disputes

“The joint forum has come a long way since the spring – decision rights on which alerts the agents may close without a reviewer are now written down and both sides work to the same list, and that argument used to take two weeks each time it came up. What has not moved is visibility of changes. Twice this quarter a tuning change on our side went live before the bank's conduct team knew of it, and while neither was harmful, we would have no fast way to pull an agent back mid-shift if one ever were. Kindly note that is a build neither side has funded.” -Portfolio Director, CXM, Regulatory, audit, tax and legal

Part 2 (Co-Management, section 3.2) reports the association of the visibility of updates with the four study measures.

3.3 Joint Governance of Agent Behavior Change Between Formal Releases

Indicator C146 · Rate how well the two firms jointly govern changes agents make to their own behavior between formal releases · Full statistical analysis ›

Weak joint governance of behavior changes between releases was associated with low scores. This extends the change-control question beyond formally scheduled updates to behavior that changes while the system remains in operation. A release process can be well defined while this intervening responsibility remains unclear. The indicator adds a distinct control question even though its statistical pattern resembles those of other co-management indicators.

Of the 388 respondents who rated it 1 to 3, 253 had a score of 3 or below, the same count and share as for step ownership. Among the 382 who rated it 4 to 7, 15 (3.93%) had a score of 3 or below, and none of the 36 who rated it 8 to 10 did (chi-square = 344.45, p < 0.001, Cramér's V = 0.65).

The governance of change between releases was associated with the score at r = 0.90 (95% CI 0.87 to 0.93, n = 150 engagements), the joint weakest of the 20, and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.23 (p = 0.004).

The association was observed within the canceled engagements at r = 0.75 and across the active stages at 0.80 to 0.84, and after Holm adjustment it held in 86 of the 87 segment levels tested. Within role it stood at 0.86 among individual contributors and portfolio directors and at 0.85 among engagement managers. Among individual contributors alone, 67.20% (211 of 314) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.70 (n = 388), against 5.22 (n = 418) among the rest (cluster-robust p < 0.001).

Fourteen respondents on 14 engagements wrote about change between releases, and 12 of the answers were negative, 4 from canceled and 8 from active engagements. Three of the 12 described one of the 26 forms of a control bypassed or not working. Respondents described agent behavior that drifted between releases with no forum quick enough to rule on whether a drift was a fix or a defect, so that the delivery team decided and informed both firms afterward. An individual contributor reported change control faster than on earlier work and no agreed definition of a defect when a configuration retuned itself.

“The scale does not really capture timing. On paper the joint model works — shared risk log, weekly review, both sides turn up prepared, and the quality checks do catch the errors the agents actually make. What it cannot show is that agent behavior drifts a little between releases and there is no forum that meets fast enough to rule on whether a drift is a fix or a defect. We end up deciding on the desk and telling both sides afterward.” -Engagement Manager, CXM, Client servicing and communications

“Both sides turn up when a Feed goes bad, the reviewing keeps pace with what the agents produce, moreover change control is faster here than what i saw before. Where it slips is we dont have an agreed definition of a defect when the self healing config retunes itself, so it sits in two trackers and each side thinks the other one has it” -Individual Contributor, ITO, Technology engineering and operations

Part 2 (Co-Management, section 3.3) reports the association of the governance of change between releases with the four study measures.

4 Human Review and the Joint Quality Assurance Loop

Human review was associated with the wider score across placement, capacity, scrutiny, error capture, and measurement. These indicators distinguish a formal review point from a review system that can examine the relevant volume of work and identify consequential errors. Their common pattern supports examining the operation of the whole review arrangement. Each indicator remains necessary because a weakness in one dimension is not described by the others alone.

“the problem is review is only a signature now. risk register did get better, we go through it together every fortnight and both sides see same list. but on code review two people tick it in ten minutes and nobody checks what the agent actually changed” -Individual Contributor, ITO, Technology engineering and operations

4.1 Placement of Human Review Points

Indicator C147 · Rate how well human review points are placed across the two firms at the moments where agent errors would matter most · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 6.2 · Full statistical analysis ›

Poor placement of human review points was associated with low scores. Review contributes to control only if it occurs where an error can still be identified and addressed before its consequences matter. This indicator concerns that placement, rather than the number of reviewers or the scrutiny they apply. Its rating-band comparisons describe the reported relationship and do not validate a boundary between working and failed governance.

Scores were 3 or below for 253 of 393 (64.38%) at indicator ratings 1–3. The corresponding counts were 10 of 378 (2.65%) at indicator ratings 4–7 and 0 of 26 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 345.33, p < 0.001, Cramér's V = 0.66.

Placement was associated with the score at r = 0.91 (95% CI 0.88 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.77. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.23 (p = 0.004).

The association was observed within the canceled engagements at r = 0.78 and ranged across the active-stage groups from 0.77 in Build-Out to 0.85 in Production. After Holm adjustment, it was significant in 85 of the 87 segment levels tested. Within role it stood at 0.86 among individual contributors and engagement managers and at 0.88 among portfolio directors. Among individual contributors alone, 65.83% (210 of 319) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.72 (n = 393), against 5.28 (n = 404) among the rest (cluster-robust p < 0.001).

Placement tracked the manual alteration of AI outputs across its rating bands. Of the respondents who rated placement 1 to 3, 52.65% (199 of 378) altered AI outputs, against 13.98% (52 of 372) at ratings of 4 to 7 and 3 of 25 at ratings of 8 to 10 (chi-square p < 0.001). Part 5 (Leading Dynamics of Manual Alteration, section 6.2) holds the crossing of placement with manual alteration in full.

Placement drew 105 written answers from 103 respondents on 75 engagements, and 75 were negative, 30 from canceled and 45 from active engagements. Of the 75, 14 (18.67%) described one of the 26 forms of a control bypassed or not working, most often problems raised with no response (9) and change discovered only through failure (4). Respondents described review that sat after validation or after the output had left, tickets that closed with neither firm checking them, and an agent that rewrote consent flags on nine thousand records overnight without approval.

“Neither side checks a ticket before it closes. Thats all.” -Individual Contributor, CXM, Corporate and back-office functions

“In March the agent rewrites consent flags on nine thousand records overnight. You are asking me about accuracy and memory, but the thing that mattered was it acted at all, without anyone aproving it.” -Engagement Manager, ITO, Technology engineering and operations

Part 2 (Co-Management, section 4.1) reports the association of placement with the four study measures.

4.2 Human Review Capacity Against Agent Output Volume

Indicator C148 · Rate how well joint human review capacity keeps up with the volume of output the agents produce · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 6.3 · Full statistical analysis ›

Insufficient human review capacity was associated with low scores. The control issue is whether the volume of agent output can be reviewed by the people assigned to oversee it. Capacity differs from review placement and from genuine scrutiny: a correctly placed review point may still be overwhelmed. The written accounts and ratings address that operational mismatch, while the score association does not identify a sufficient staffing level.

Of the 396 respondents who rated review capacity 4 to 7, 21 (5.30%) had a score of 3 or below. Among the 381 who rated it 1 to 3, 244 (64.04%) had a score of 3 or below, and none of the 28 who rated it 8 to 10 did (chi-square = 317.63, p < 0.001, Cramér's V = 0.63).

Review capacity was associated with the score at r = 0.90 (95% CI 0.86 to 0.92, n = 150 engagements), the joint weakest of the 20, and with governance in the other four categories at r = 0.75, the weakest of the 20. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.25 (p = 0.003).

The association was observed within the canceled engagements at r = 0.76 and ranged across the active-stage groups from 0.75 in Build-Out to 0.82 in Transition/Cut-Over and Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.86 among individual contributors and 0.83 among engagement managers and portfolio directors. Among individual contributors alone, 66.03% (206 of 312) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.71 (n = 381), against 5.18 (n = 424) among the rest (cluster-robust p < 0.001).

Review capacity tracked human rework. Of the respondents who rated capacity 1 to 3, 43.57% (166 of 381) described human rework absorbing agent failure, against 14.90% (59 of 396) at ratings of 4 to 7 and 1 of 28 at ratings of 8 to 10 (chi-square p < 0.001). The written pattern of review overrun by agent volume is analyzed with the other reasons for reported governance weakness under The Fit Between Governance Processes and the Pace of Agent Work.

Review capacity drew 145 written answers from 135 respondents on 89 engagements, and 134 of them were negative, 42 from canceled and 92 from active engagements. Of the 134, 17 (12.69%) described one of the 26 forms of a control bypassed or not working, most often unclear liability or regulatory accountability (6). Respondents described review that grew shorter as volume grew, a formal quality sample of about two percent, and one or two people as the whole of review across several workstreams.

“I edit the suggested response before the Agent sends it, it still guesses wrong on routing a lot. And QA on this is basically me reading things. There is a formal QA step, it samples about two percent, which means ninety eight percent of what goes out has had exactly one set of eyes on it and those eyes are mine and I am also handling my own queue. That is the honest picture. Nobody is hiding it, its just that nobody has added it up. The agent handles the easy ones well, genuinely, and for the routine balance and payment questions its better than we were. Its the edge cases where it confidently gets it wrong, and confidently is the part that matters, because it does not flag uncertainty, it just answers. So you have to read all of them to find the few. And the productivity numbers we report do not account for the reading. I have brought this up with my supervisor twice. Both times the response was that the sampling rate would be reviewed. I do not think anyone is being” -Individual Contributor, CXM, Client servicing and communications

“one reviewer for forty provisioning outputs in a day, she cant” -Individual Contributor, ITO, Technology engineering and operations

Part 2 (Co-Management, section 4.2) reports the association of review capacity with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 6.3) its association with manual alteration.

4.3 Genuine Scrutiny at Human Review Points

Indicator C149 · Rate how much genuine scrutiny agent outputs receive at human review points rather than routine approval · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 6.4 · Full statistical analysis ›

Routine approval without substantial scrutiny was associated with a weaker reported control environment. The indicator asks whether human review operates as an examination of agent work or primarily as a procedural sign-off. It gives the governance analysis a quality dimension that review placement and capacity do not capture alone. Differences among subgroup correlations remain descriptive and do not establish that scrutiny deteriorated on cancellation.

Within canceled engagements, the correlation was 0.74, jointly the lowest of the 20 with step ownership, while across the active stages it ran at 0.81 in Build-Out, 0.83 in Transition/Cut-Over, and 0.86 in Production.

Scores were 3 or below for 251 of 393 (63.87%) at indicator ratings 1–3. The corresponding counts were 15 of 370 (4.05%) at indicator ratings 4–7 and 0 of 31 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 322.30, p < 0.001, Cramér's V = 0.64. Scrutiny was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.78. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.25 (p = 0.002).

After Holm adjustment the association held in 86 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.86 among engagement managers, and 0.83 among portfolio directors. Among individual contributors alone, 65.62% (208 of 317) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.71 (n = 393), against 5.27 (n = 401) among the rest (cluster-robust p < 0.001).

Genuine scrutiny drew 66 written answers from 63 respondents on 50 engagements, and 59 were negative, 30 from canceled and 29 from active engagements. Of the 59, 11 (18.64%) described one of the 26 forms of a control bypassed or not working, most often activity counted in place of outcome (4), change discovered only through failure (3), and problems raised with no response (3). Respondents described releases approved without reading, approval by the same person who had prepared the case, and a queue of forty offers from which thirty-eight were released.

“One issue dominates and I will leave the others alone. The human review point on arrangement offers has quietly become an approval step rather than a scrutiny step. The reviewer opens a queue of forty and releases thirty-eight, which is not review in any sense a supervisor would accept. That is a capacity design problem and not a people problem. The governance we jointly operate was drawn up for a world in which a person made each offer and could be asked afterwards why, and it does not sit comfortably over work of this shape, however well both sides chair the meetings.” -Portfolio Director, CXM, Payments, cards and disputes

“Nobody asked this one, the person who approve our HR cases is the same person who prepare them, so the review is only a click. Please note also when they change the routing rules nobody tell us, we find out from the failed cases” -Individual Contributor, BPO, Corporate and back-office functions

Part 2 (Co-Management, section 4.3) reports the association of scrutiny with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 6.4) its association with manual alteration.

4.4 Reliability of the Joint Quality Assurance Loop

Indicator C150 · Rate how reliably the joint quality assurance loop catches agent errors before they reach live operations · Full statistical analysis ›

A less reliable joint quality assurance loop was associated with low scores. This concerns whether the combined process catches agent errors before they reach live operations, connecting review activity with its reported operational result. It differs from the existence of a review step or a formal quality policy. The adjusted association remains conditional on the comparison measure, with no inference that its relative ranking measures the control's practical importance.

The quality loop was associated with governance in the other four categories at r = 0.77, and after adjustment for the mean of the other 53 indicators the association remained at partial r = 0.21 (p = 0.012). It was associated with the score at r = 0.91 (95% CI 0.88 to 0.94, n = 150 engagements).

The weak adjusted association sat beside one of the sharpest band crossings of the 20. Scores were 3 or below for 244 of 377 (64.72%) at indicator ratings 1–3. The corresponding counts were 13 of 381 (3.41%) at indicator ratings 4–7 and 0 of 29 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 338.49, p < 0.001, Cramér's V = 0.66.

The association was observed within the canceled engagements at r = 0.77 and ranged across the active-stage groups from 0.78 in Build-Out to 0.86 in Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.86 among individual contributors and engagement managers and at 0.76 among portfolio directors. Among individual contributors alone, 67.55% (204 of 302) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.71 (n = 377), against 5.22 (n = 410) among the rest (cluster-robust p < 0.001).

The written base was thin. Twenty-nine respondents on 28 engagements wrote about the quality loop, and 22 of the answers were negative, 11 from canceled and 11 from active engagements. Of the 22, 7 (31.82%) described one of the 26 forms of a control bypassed or not working, most often change discovered only through failure (3) and problems raised with no response (3). Respondents described errors found only once they were live, and drafts that went out wrong without anyone stopping them.

“I give 1 for one reason, no one ever stop a draft going out wrong” -Individual Contributor, CXM, Client servicing and communications

“My 4 is only for the QA bit, the errors we find are allready live by then” -Individual Contributor, CXM, Client servicing and communications

Part 2 (Co-Management, section 4.4) reports the association of the quality loop with the four study measures.

4.5 Quality Measures That Capture Agent Errors

Indicator C151 · Rate how well the quality measures both firms use capture the kinds of errors agents actually make · Full statistical analysis ›

Quality measures that inadequately captured agent errors were associated with low scores. The measures used by both firms need to capture the errors agents actually make. Counts of completed work or cleared exceptions can be informative while leaving incorrect completions poorly represented. The limited written accounts illustrate that distinction, without establishing an exhaustive inventory of errors or the independent accuracy of the measures.

The indicator was associated with the score at r = 0.93 (95% CI 0.90 to 0.95, n = 150 engagements), third among the 47 governance indicators. It was associated with governance in the other four categories at r = 0.79, and after adjustment for the mean of the other 53 indicators the association remained at partial r = 0.26 (p = 0.002), the fifth-strongest of the 20.

Scores were 3 or below for 250 of 405 (61.73%) at indicator ratings 1–3. The corresponding counts were 12 of 357 (3.36%) at indicator ratings 4–7 and 0 of 30 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 307.41, p < 0.001, Cramér's V = 0.62. The 1 to 3 band share was the second-lowest of the 20, the pattern seen for accountability, where a close tie across the whole range sat beside low ratings spread more widely across engagements.

The association was observed within the canceled engagements at r = 0.77 and across the active stages at 0.81 to 0.85, and after Holm adjustment it held in 85 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors, 0.88 among engagement managers, and 0.86 among portfolio directors. Among individual contributors alone, 63.53% (209 of 329) of those who rated them 1 to 3 had a score of 3 or below. Respondents who rated them distressed recorded a score of 2.73 (n = 405), against 5.33 (n = 387) among the rest (cluster-robust p < 0.001).

Fourteen respondents on 14 engagements wrote about quality measures, and 11 of the answers were negative, 6 from canceled and 5 from active engagements. Five of the 11 described one of the 26 forms of a control bypassed or not working, most often change discovered only through failure (3). The answers placed the failure in measures designed for the errors people make. Those measures counted an exception cleared as a pass whether or not it had been cleared correctly. An individual contributor noticed an unannounced change to an audit sampling rule only when the exception count halved overnight.

“There was a sampling rule that changed on a continuous audit run and nobody on either side told us, i only noticed because exception count dropped by half overnight and that is not normal for that control. When i asked, neither side would say who approved it. Their audit lead said it is vendor change, my lead said it is their side, and i am still not knowing which is true. On my previous engagement, which was reconciliation work, this was different. Not perfect but there was one change log and both firms wrote into it, so if something moved you could at least see who moved it and when. Here there is nothing like the same. Changes just appear. And second thing is the qa checklist, it is written for a human tester. It asks did you check the sample, did you sign it, it does not ask did the tool behave the way it behaved last week, because nobody thought to put that question in. So we tick boxes that mean nothing and the real problem goes straight through. Kindly note i did raise this with my lead in Janurary. Nothing came back. Previous engagement was a small team and a decision took a day, here a decision needs three names and it takes a month or it never comes at all, and then we just decide ourself and carry on, which everybody knows and nobody writes down.” -Individual Contributor, KPO, Regulatory, audit, tax and legal

Part 2 (Co-Management, section 4.5) reports the association of quality measures with the four study measures.

5 The Shift From Managing People to Managing Agents

The fit between co-management and agent-performed work was associated with the wider score. Cross-firm visibility, decision speed, present operating-model fit, and readiness for expected autonomy describe different demands on that management arrangement. The evidence connects those reported demands with governance quality, while the future-autonomy item remains an expectation rather than observed future capability.

5.1 Each Firm's Visibility of the Other Firm's Agent Actions

Indicator C152 · Rate how fully each firm can see what the other firm's agents actually did when questions arise · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 6.7 · Full statistical analysis ›

Limited visibility of the other firm's agent actions was associated with low scores. Oversight requires an account of what occurred across the shared workflow when a question or incident arises. This indicator concerns cross-firm access to that account, beyond whether a tool generates logs internally. The association was observed within active and canceled groups, with no direct test of how visibility changed over time.

The association within the canceled engagements matched the association across the active stages, at 0.82 in Build-Out, 0.84 in Transition/Cut-Over, and 0.83 in Production, with no direct test of change over time. After Holm adjustment, it was significant in 86 of the 87 segment levels tested.

Scores were 3 or below for 245 of 392 (62.50%) at indicator ratings 1–3. The corresponding counts were 11 of 362 (3.04%) at indicator ratings 4–7 and 0 of 26 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 314.93, p < 0.001, Cramér's V = 0.64. Visibility was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.79. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.22 (p = 0.007).

Within role the association stood at 0.88 among individual contributors, 0.87 among engagement managers, and 0.85 among portfolio directors. Among individual contributors alone, 64.86% (203 of 313) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.74 (n = 392), against 5.35 (n = 388) among the rest (cluster-robust p < 0.001).

Visibility drew 66 written answers from 64 respondents on 49 engagements, and 62 of them were negative, 23 from canceled and 39 from active engagements, the most uniformly negative testimony among the co-management indicators. Of the 62, 9 (14.52%) described one of the 26 forms of a control bypassed or not working, most often unclear liability or regulatory accountability (3) and exceptions omitted from the reports the financial institution received (2). Respondents described agent actions reconstructed from logs after the event and answers that were, by the time they were given, partly guessed.

“I will say this for the joint side, when we do get their AML lead in a room the decisions are quick and they hold, and their reviewers keep up with the volume we send, which I did not expect. The failure is not the people, it is that neither firm has sight of the other side's agent actions. They ask me on Tuesday why an alert closed on Sunday and I reconstruct it from logs, and by then the answer is half a guess. Nobody designed for that question.” -Engagement Manager, BPO, Financial crime and client onboarding

“When something the agent did needs explaining i cant see there side of it, so i rebuild the break from the custodian statement myself and work back from that. Takes a morning. My team does the same. Nobody reviews the volume that comes out, we just spot check the big ones” -Individual Contributor, BPO, Trade, settlement and reconciliation

Part 2 (Co-Management, section 5.1) reports the association of visibility with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 6.7) its association with manual alteration.

5.2 Joint Decision Speed Against Agent Speed

Indicator C153 · Rate how closely the speed of joint decision making matches the speed at which the agents act · Full statistical analysis ›

Joint decisions that did not keep pace with agent action were associated with low scores. The governance issue is the time available to decide when operational work continues faster than the two firms can reach agreement. This differs from deciding who has authority in principle. The evidence describes reported decision speed and its relationship with the broader score, not a measured delay for every decision.

Of the 381 respondents who rated joint decision speed 1 to 3, 245 (64.30%) had a score of 3 or below. Among the 379 who rated it 4 to 7, 12 (3.17%) had a score of 3 or below, and none of the 26 who rated it 8 to 10 did (chi-square = 335.79, p < 0.001, Cramér's V = 0.65).

Decision speed was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.79. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.23 (p = 0.005).

The association was observed within the canceled engagements at r = 0.76 and ranged across the active-stage groups from 0.82 in Build-Out and Transition/Cut-Over to 0.87 in Production. After Holm adjustment, it was significant in 85 of the 86 segment levels tested. Within role it stood at 0.87 among individual contributors and engagement managers and at 0.90 among portfolio directors. Among individual contributors alone, 66.67% (202 of 303) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.69 (n = 381), against 5.28 (n = 405) among the rest (cluster-robust p < 0.001).

Fourteen respondents on 13 engagements wrote about decision speed, and 13 of the answers were negative, 7 from canceled and 6 from active engagements. Three of the 13 described problems raised with no response. The answers set the hour in which an agent acted against the days that two people took to agree on what it had done, and a weekly meeting against a queue the agent worked every hour.

Part 2 (Co-Management, section 5.2) reports the association of decision speed with the four study measures. The pace of governance against the pace of the agents is analyzed as a reason for reported governance weakness under The Fit Between Governance Processes and the Pace of Agent Work.

5.3 Fit of the Joint Governance Model to Work Performed by Agents

Indicator C154 · Rate how well today's joint governance model fits work performed by agents rather than work performed by people · Full statistical analysis ›

A joint model poorly fitted to agent-performed work was associated with low scores. The concern is whether authority, review, and issue management are designed for the work now being performed, rather than carried over without adaptation from people-based delivery. The small written base illustrates reported gaps in authority. It does not establish that all arrangements inherited from earlier work are ineffective.

Of the 395 respondents who rated the fit of the joint model to agent work 1 to 3, 251 (63.54%) had a score of 3 or below. Among the 369 who rated it 4 to 7, 13 (3.52%) had a score of 3 or below, and none of the 39 who rated it 8 to 10 did (chi-square = 331.52, p < 0.001, Cramér's V = 0.64).

The fit of the model was associated with the score at r = 0.92 (95% CI 0.89 to 0.94, n = 150 engagements) and with governance in the other four categories at r = 0.78. After adjustment for the mean of the other 53 indicators, the association remained at partial r = 0.26 (p = 0.001), the fourth-strongest of the 20.

The association was observed within the canceled engagements at r = 0.77 and across the active stages at 0.81 to 0.85, and after Holm adjustment it held in 86 of the 87 segment levels tested. Within role it stood at 0.87 among individual contributors and engagement managers and at 0.80 among portfolio directors. Among individual contributors alone, 65.83% (210 of 319) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.73 (n = 395), against 5.24 (n = 408) among the rest (cluster-robust p < 0.001).

Ten respondents on 9 engagements wrote about the fit of the model, and 9 of the answers were negative, 5 from canceled and 4 from active engagements. Three of the 9 described one of the 26 forms of a control bypassed or not working. The answers described leads on both sides who held no authority to decide, and a model that worked issues as one team while keeping no joint record of what the agents had changed.

“To be fair to the leads on both sides, none of them were given any power to decide. They sit in the same queue we do… every exception we raise goes up and stops somewhere above them. I don't think they wanted it this way, they simply had nothing to work with.” -Individual Contributor, BPO, Corporate and back-office functions

“Issues do get worked as one team, that part functions. The rest is built for people doing the work. There is no joint record of what the agents changed between reviews, so when a forecast number moves we cannot say which side moved it. It gets discussed and then it repeats next cycle” -Engagement Manager, KPO, Corporate and back-office functions

Part 2 (Co-Management, section 5.3) reports the association of the fit of the joint model with the four study measures. The inheritance of governance functions designed for people is analyzed as a reason for reported governance weakness under The Fit Between Governance Processes and the Pace of Agent Work.

5.4 Readiness of the Co-Management Model for Expected Agent Autonomy

Indicator C155 · Rate how ready the joint management model is for the level of agent autonomy you expect in this engagement within two years · Full statistical analysis ›

Reported readiness for expected agent autonomy was strongly associated with the wider score. This places the anticipated expansion of autonomous work within the governance question: the joint model must be assessed against the authority agents are expected to exercise. The measure concerns respondents' expectations within two years, not observed future performance. Its high correlation is evidence of aligned assessments, not proof that future autonomy will be governed successfully.

The association with the score stood at r = 0.93 (95% CI 0.90 to 0.95, n = 150 engagements). Readiness was associated with governance in the other four categories at r = 0.80, the highest of the 20, and after adjustment for the mean of the other 53 indicators the association remained at partial r = 0.22 (p = 0.008).

Scores were 3 or below for 254 of 407 (62.41%) at indicator ratings 1–3. The corresponding counts were 14 of 373 (3.75%) at indicator ratings 4–7 and 0 of 28 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 316.46, p < 0.001, Cramér's V = 0.63. Respondents who rated it distressed recorded a score of 2.70 (n = 407), against 5.30 (n = 401) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.76 and ranged across the active-stage groups from 0.83 in Build-Out to 0.88 in Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.88 among individual contributors, 0.87 among engagement managers, and 0.82 among portfolio directors. Among individual contributors alone, 63.20% (213 of 337) of those who rated it 1 to 3 had a score of 3 or below.

Three respondents on 3 active engagements wrote about readiness for expected autonomy, and two of the answers were negative. Neither negative answer described one of the 26 forms of a control bypassed or not working. One individual contributor judged that the joint model would not hold at the autonomy expected within two years, and a portfolio director on another engagement reported that the conversation on which alert types an agent might close alone had been deferred twice.

Part 2 (Co-Management, section 5.4) reports the association of readiness with the four study measures.

The vendor's incident log classified access errors as customer-caused whenever the underlying policy data came from the customer's systems, even when the agent's own logic misapplied that policy.”

INDIVIDUAL CONTRIBUTOR · ITO, Technology engineering and operations