Third-Party Dependencies and the Limits of Internal Control

External dependencies were associated with the broader control environment through notice, monitoring, contractual coverage, security, and operational knowledge. The ten third-party governance indicators had 2,533 low ratings among 7,944 ratings. The governance, oversight, and control score relates these dependencies to the engagement's other controls. Written accounts of change discovered through failure add a specific detection concern, while adjusted associations remain conditional and less conclusive.

Among respondents who rated a third-party control 1 to 3, 73.41% to 80.41% had a score of 3 or below, the highest range of the five categories. Ratings of 4 to 7 still carried a score of 3 or below for 14.14% to 17.03% of respondents, and the upper end of that range was the highest of the 47 governance indicators. Each indicator was associated with governance in the other four failure categories at r = 0.75 to 0.78 (n = 150 engagements), and after adjustment for the mean of the other 53 indicators the partial r ran from −0.14 to −0.04, with none of the ten significant.

Written accounts of change discovered through failure occurred on 93 of the 150 engagements and included six of the 39 respondents above score 7. Respondent mean scores were not clearly different between those describing that pattern and those not describing it. Unannounced schema and format changes similarly showed no clearly detected mean contrast. These results preserve a specific detection concern across the observed score range, without demonstrating equal incidence. Table 3.12 reports the ten controls and two written patterns.

On my last job the network sent a bulletin six weeks before a mandate change and you planned around it. Here we found out from a failed auth response. Same networks. The difference is nobody set up the notifications.”

INDIVIDUAL CONTRIBUTOR · ITO, Payments, cards and disputes

1 Notice of Provider Change

Provider notice and delivery of that notice to operators were associated with the wider score. They are separate control steps: a provider may communicate a change without the people running the affected agents receiving or acting on it. The written accounts place part of the notice problem within the engagement's own arrangements. This distinction gives operational meaning to the ratings beyond a simple measure of provider communication quality.

1.1 Quality of Advance Notice From Third-Party Providers

Indicator C181 · Rate the quality of advance notice third-party providers give before changes that can affect the agents on this engagement · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.4 · Full statistical analysis ›

Poor advance notice of provider change was associated with low scores. This places an external dependency inside the governance analysis: the financial institution and vendor need time to assess a change that can affect their agents. Notice quality is distinct from whether the message reaches operators and from whether the engagement acts on it. The indicator measures reported notice rather than every provider change event.

Of the 250 respondents who rated advance notice 1 to 3, 192 had a score of 3 or below. Among the 466 who rated it 4 to 7, 70 (15.02%) did, and none of the 85 who rated it 8 to 10 did (chi-square = 328.36, p < 0.001, Cramér's V = 0.64).

Advance notice was associated with the score computed without it at r = 0.85 (95% CI 0.80 to 0.89, n = 150 engagements) and with governance in the other four categories at r = 0.77. After adjustment for the mean of the other 53 indicators, the partial r was −0.09 (p = 0.277). Respondents who rated it distressed recorded a score of 2.33 (n = 250), against 4.76 (n = 551) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.72 and across the active stages at 0.73 in Build-Out, 0.69 in Transition/Cut-Over, and 0.78 in Production. After Holm adjustment, it was significant in 85 of the 87 segment levels tested. Within role it stood at 0.82 among individual contributors, 0.77 among engagement managers, and 0.76 among portfolio directors. Among individual contributors alone, 77.93% (166 of 213) of those who rated it 1 to 3 had a score of 3 or below.

Advance notice drew 160 written answers from 159 respondents on 96 engagements, and 108 were negative, 53 from canceled and 55 from active engagements. Of the 108, 83 (76.85%) described one of the 26 forms of a control bypassed or not working, most often change discovered only through failure (60) and schema or format changed without notice (41). Among the 52 answers that were not negative, several credited the providers with notice and placed the failure on the engagement's own side, where no person was assigned to read the notices that arrived.

“to be fair the screening provider does send notice when their list format change. nobody on our side is given the job of reading it, and so we find out when a batch fail. that part is not on them” -Individual Contributor, BPO, Financial crime and client onboarding

“To be fair to the providers they do send the deprecation notices well in advance, we just had nobody free to action them. The bigger problem is half their systems only have a screen, no proper api, so the agent has to click like a person” -Individual Contributor, ITO, Technology engineering and operations

Part 2 (Third-Party Ecosystem, section 1.1) reports the association of advance notice with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 7.4) its association with manual alteration.

1.2 Delivery of Change Notices to the Operators of Affected Agents

Indicator C182 · Rate how reliably word of an upcoming third-party change reaches the people who operate the affected agents in time to act · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.8 · Full statistical analysis ›

Failure to deliver change notices to affected operators was associated with low scores. This identifies a control step inside the engagement even when a third-party provider gives notice. Information must reach people who can assess the effect on agent work before the change takes effect. The finding separates the existence of communication from its timely operational use.

Within canceled engagements, the correlation was 0.65, below its value in each active stage, at 0.78 in Build-Out, 0.70 in Transition/Cut-Over, and 0.74 in Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested.

Scores were 3 or below for 188 of 255 (73.73%) at indicator ratings 1–3. The corresponding counts were 78 of 467 (16.70%) at indicator ratings 4–7 and 0 of 76 (0.00%) at indicator ratings 8–10. The reported band results were chi-square = 283.34, p < 0.001, Cramér's V = 0.60. The indicator was associated with the score computed without it at r = 0.83 (95% CI 0.78 to 0.88, n = 150 engagements) and with governance in the other four categories at r = 0.75, the lowest of the ten. After adjustment for the mean of the other 53 indicators, the partial r was −0.11 (p = 0.168).

Within role the association stood at 0.79 among individual contributors, 0.80 among engagement managers, and 0.84 among portfolio directors. Among individual contributors alone, 76.44% (159 of 208) of those who rated it 1 to 3 had a score of 3 or below. Respondents who rated it distressed recorded a score of 2.37 (n = 255), against 4.72 (n = 543) among the rest (cluster-robust p < 0.001).

The indicator drew 56 written answers from 55 respondents on 47 engagements, and 42 were negative, 22 from canceled and 20 from active engagements. Of the 42, 24 (57.14%) described one of the 26 forms of a control bypassed or not working, most often change discovered only through failure (18). Respondents described notices that the providers sent and that never reached the team operating the agents, and a delivery team that learned of a change from its own logs.

“Last release the market data vendor changed a price field type and our pricing module threw errors all morning. notice was sent apparently, it just never reached my team, we found it from the logs ourself” -Individual Contributor, ITO, Technology engineering and operations

“The API people are not the villains here. They publish their changes, it is our own side that never passes it down to the floor… not their fault.” -Individual Contributor, CXM, Client servicing and communications

Part 2 (Third-Party Ecosystem, section 1.2) reports the association of the delivery of notices with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 7.8) its association with manual alteration.

2 Detection of Unannounced Change

Protection and monitoring were associated with the wider score, while written descriptions identified changes discovered only after failure. This brings detection into the governance question. Advance notice cannot account for changes that arrive without warning, so the engagement's ability to identify disruption remains a separate responsibility. The written comparisons do not establish that late discovery was equally prevalent across score levels.

2.1 Protection of Agents From Silent Failure

Indicator C178 · Rate how well the agents on this engagement are protected from breaking silently when a third party changes something without warning · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.5 · Full statistical analysis ›

Weak protection against silent breakage was associated with low scores. The governance issue is whether external change can disrupt agent work without timely detection. This differs from advance notice, because protection is still needed when providers do not warn the engagement. The rating concerns perceived protection and does not count all silent failures or establish their independently observed frequency.

Of the 249 respondents who rated protection 1 to 3, 186 had a score of 3 or below. Among the 456 who rated it 4 to 7, 74 (16.23%) did, and none of the 76 who rated it 8 to 10 did (chi-square = 289.95, p < 0.001, Cramér's V = 0.61).

Protection was associated with the score computed without it at r = 0.85 (95% CI 0.80 to 0.89, n = 150 engagements) and with governance in the other four categories at r = 0.78, the highest of the ten. After adjustment for the mean of the other 53 indicators, the partial r was −0.06 (p = 0.457). Respondents who rated it distressed recorded a score of 2.40 (n = 249), against 4.76 (n = 532) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.70 and across the active stages at 0.74 to 0.77, and after Holm adjustment it held in 85 of the 87 segment levels tested. Within role it stood at 0.82 among individual contributors, 0.81 among engagement managers, and 0.78 among portfolio directors. Among individual contributors alone, 74.64% (156 of 209) of those who rated it 1 to 3 had a score of 3 or below.

Protection drew 136 written answers from 134 respondents on 81 engagements, and 100 were negative, 50 from canceled and 50 from active engagements. Of the 100, 74 described one of the 26 forms of a control bypassed or not working, and 66 of them described change discovered only through failure. Respondents described provider releases over a weekend that left agents processing nothing, or processing wrongly, until a person noticed on the next working day.

“the ticketing tool pushed a new api version on a saturday. our agent kept closing change tickets that were never raised, and nobody noticed till monday cutover. the old release guy used to catch that by eye” -Individual Contributor, ITO, Technology engineering and operations

“The OCR vendor pushed a version change over a weekend and no one told us… we found it Monday when half the scans came back with blank fields.” -Individual Contributor, BPO, Corporate and back-office functions

Part 2 (Third-Party Ecosystem, section 2.1) reports the association of protection with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 7.5) its association with manual alteration.

2.2 Monitoring for Unannounced Third-Party Change

Indicator C183 · Rate the engagement's monitoring for unannounced changes in the third-party APIs, data feeds, and models its agents depend on · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.12 · Full statistical analysis ›

Weak monitoring for unannounced provider change was associated with low scores. This control asks whether the engagement can detect changes in the interfaces, data feeds, and models on which agents depend. It extends oversight beyond a reliance on release notices. The adjusted model did not detect a clear separate cross-category association, while the written accounts supply context about how changes become visible.

Of the 251 respondents who rated monitoring 1 to 3, 188 had a score of 3 or below. Among the 465 who rated it 4 to 7, 75 (16.13%) did, and none of the 78 who rated it 8 to 10 did (chi-square = 297.02, p < 0.001, Cramér's V = 0.61).

Monitoring was associated with the score computed without it at r = 0.84 (95% CI 0.79 to 0.88, n = 150 engagements) and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the partial r was −0.12 (p = 0.136). Respondents who rated it distressed recorded a score of 2.40 (n = 251), against 4.72 (n = 543) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.69 and across the active stages at 0.71 to 0.75, and after Holm adjustment it held in 84 of the 87 segment levels tested. Within role it stood at 0.81 among individual contributors, 0.76 among engagement managers, and 0.75 among portfolio directors. Among individual contributors alone, 77.56% (159 of 205) of those who rated it 1 to 3 had a score of 3 or below.

Forty-six respondents on 39 engagements wrote about monitoring, and 30 of the answers were negative, 12 from canceled and 18 from active engagements. Of the 30, 22 described one of the 26 forms of a control bypassed or not working, 21 of them change discovered only through failure. Respondents described no alarm on the third-party side, so that employees reported a failure before any system did. Where monitoring had been built, it caught a change within the hour and left the work it disrupted to be done manually.

“Ask anyone here, the third party side has no alarm on it, we find out when the bot start failing, employees tell us before any system do” -Individual Contributor, CXM, Corporate and back-office functions

“Our transcription provider changed its speaker labelling format in June without telling anyone, and our own monitoring caught it inside forty minutes, which is why this block scores well from me. What ratings do not show is what came after, two analysts labelled calls by hand for a day and a half while mapping was corrected, and if volume had been three times larger we simply could not have done it. Monitoring here is genuinely strong, contracts with these providers are much weaker, none of them owes us notice of anything.” -Engagement Manager, CXM, Credit, underwriting and suitability

Part 2 (Third-Party Ecosystem, section 2.2) reports the association of monitoring with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 7.12) its association with manual alteration.

2.3 Changes Discovered Only Through Failure

Written pattern · All written answers to the open questions · 157 answers from 150 respondents on 93 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.7

Discovering provider change through failure was reported across a range of scores. Detection only after disruption leaves the engagement responding after a provider change has affected operations. Its written reach and the score comparisons describe different questions, so the number of engagements mentioning the pattern cannot establish its full incidence. The lack of a clear adjusted association does not make late discovery an adequate control arrangement.

The 150 respondents who described it stood at 3.82 and the other 700 at 4.03 (cluster-robust p = 0.207). Among individual contributors alone, the 113 who described it stood at 3.73 against 3.89 among the other 554 (cluster-robust p = 0.369).

At the engagement level, the share of respondents describing the pattern was associated with the score at r = −0.17 (p = 0.042, n = 150 engagements). After adjustment for the mean of the other 53 indicators the association fell to partial r = −0.04 (p = 0.657). The adjusted comparison did not detect a clear association beyond the comparison mean.

The pattern reached every numeric score band. It was described by 20.14% (57 of 283) of respondents at 3 or below, 16.48% (87 of 528) above 3 to 7, and 6 of the 39 above 7. It was described by 15.43% of respondents in Build-Out, 14.80% in Transition/Cut-Over, 13.59% in Production, and 22.36% on canceled engagements, and by 22.42% in the lowest indicator quintile against 17.50% in the highest. Individual contributors described it at 16.94% (113 of 667), engagement managers at 20.67% (31 of 150), and portfolio directors at 6 of 31, with no clearly detected difference by role after Holm adjustment. No segment family differed after Holm adjustment.

The 157 answers came from all written answers to the open questions, and 76 of them were written on canceled engagements. Part 5 counts 26 answers from 25 respondents on 22 engagements for the same pattern, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents described runs that failed on a Monday after a weekend release and queues that went quiet. On the highest-scoring engagement of the study, a respondent reported that the providers' change notes arrived by email and went unread until something broke.

“The rate limit is the thing people complain about but it is not really the providers fault, we send the whole book in one go at month end instead of spreading it, any provider will throttle that. Their change notes come by email every release also, we simply do not read them until something break. On the licence question i put 7 but that is because nobody has asked them yet, not because they refused” -Individual Contributor, KPO, Credit, underwriting and suitability

“On my previous account the feed providers sent change notices weeks ahead. Here we normally find out because a tick handler has already stopped” -Engagement Manager, ITO, Technology engineering and operations

Part 5 (Leading Dynamics of Manual Alteration, section 7.7) analyzes the pattern as a factor in manual alteration.

2.4 Schema and Format Changes Without Notice

Written pattern · All written answers to the open questions · 61 answers from 60 respondents on 51 engagements · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.6

Unannounced schema and format changes were described without a clearly detected difference in respondent mean scores. This identifies a specific external-change problem that a broad composite may not sharply distinguish. The governance question concerns preparation for changes to the structure agents expect, separate from general provider notice. The evidence remains a set of written descriptions, with no verified nonincident group among those who did not mention it.

The 60 respondents who described it stood at 3.70 and the other 790 at 4.01 (cluster-robust p = 0.187). Among individual contributors alone, the 51 who described it stood at 3.72 against 3.88 among the other 616 (cluster-robust p = 0.554).

At the engagement level, the share describing the pattern was associated with the score at r = −0.16 (p = 0.045, n = 150 engagements), and after adjustment for the mean of the other 53 indicators the association fell to partial r = −0.06 (p = 0.478).

The pattern was described by 8.13% (23 of 283) of respondents with a score of 3 or below, 6.63% (35 of 528) above 3 to 7, and 2 of the 39 above 7. It was described by 5.71% of respondents in Build-Out, 5.20% in Transition/Cut-Over, 6.80% in Production, and 9.32% on canceled engagements, and by 8.48% in the lowest indicator quintile against 6.25% in the highest. Individual contributors described it at 7.65% (51 of 667), engagement managers at 4.00% (6 of 150), and portfolio directors at 3 of 31, with no clearly detected difference by role after Holm adjustment. No segment family differed after Holm adjustment.

The 61 answers came from all written answers to the open questions, and 31 of them were written on canceled engagements. Part 5 counts 27 answers from 26 respondents on 24 engagements, among the written answers on the manual alteration methods alone, so the two counts rest on different bases. Respondents described a renamed column, a changed date format, or an inserted field that failed every downstream mapping at once and surfaced only at reconciliation.

“Fund administrator change their file layout in June, no notice given. We found it when reconciliation failed… Nobody monitor for this, we just see it break.” -Individual Contributor, BPO, Trade, settlement and reconciliation

“supplier risk feed changed the date format one week… nobody tell us anything. agents dropped every record and i only notice because count was too low. no email, no note, we find out ourselves” -Individual Contributor, CXM, Corporate and back-office functions

Part 5 (Leading Dynamics of Manual Alteration, section 7.6) analyzes the pattern as a factor in manual alteration.

3 Contractual and Licensing Coverage of Agent Work

Contractual coverage of disruptive provider change was associated with the wider score. This addresses a formal boundary over external dependency, while the notice and monitoring indicators address operation. The adjusted association was not clearly detected. The findings should distinguish reported contract coverage from evidence that contractual terms delivered effective operational protection.

3.1 Contractual Coverage of Disruptive Provider Change

Indicator C184 · Rate how well the contracts with third-party providers deal with provider changes that can break the agents on this engagement · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.9 · Full statistical analysis ›

Weak contractual coverage of disruptive provider change was associated with low scores. Contracts define an external responsibility boundary, but their reported coverage does not by itself show that operational notice, monitoring, or response works. The relatively higher mean rating and the large low-score share among low indicator ratings concern different quantities. Both should inform a restrained assessment of contractual control.

The mean rating stood at 4.66, level with the reliability of guardrails. Of the 256 respondents who rated contractual coverage 1 to 3, 199 had a score of 3 or below, the second-highest share of the ten. Among the 463 who rated it 4 to 7, 71 (15.33%) did, and none of the 80 who rated it 8 to 10 did (chi-square = 332.27, p < 0.001, Cramér's V = 0.64).

Contractual coverage was associated with the score computed without it at r = 0.84 (95% CI 0.79 to 0.88, n = 150 engagements) and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the partial r was −0.07 (p = 0.406). Respondents who rated it distressed recorded a score of 2.27 (n = 256), against 4.77 (n = 543) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.71 and across the active stages at 0.71 to 0.76, and after Holm adjustment it held in all 86 segment levels tested. Within role it stood at 0.82 among individual contributors, 0.78 among engagement managers, and 0.70 among portfolio directors. Among individual contributors alone, 78.87% (168 of 213) of those who rated it 1 to 3 had a score of 3 or below.

Twenty-one respondents on 21 engagements wrote about contracts, and 14 of the answers were negative, 6 from canceled and 8 from active engagements. Four of the 14 described one of the 26 forms of a control bypassed or not working. Respondents described a delivery team that held no contract with the providers its agents depended on, and contract terms that did not oblige a provider to give notice of change.

“We do not hold the contract with the HR feed provider, that is the part a score cannot show. Their directory sync breaks, leavers stay active, and all we can do is tell the bank and wait.” -Engagement Manager, ITO, Technology engineering and operations

“Dependencies are the source control platform, the artefact registry, the ticketing system and one model provider. Licensing was checked at contract stage and covers agent use, so no exposure there. Contract terms with the registry vendor do not oblige them to notify us of API changes. There is no fallback registry configured. If it goes down, releases stop until it returns; we have not tested how long that would take. Nothing to add on the rest.” -Portfolio Director, ITO, Technology engineering and operations

Part 2 (Third-Party Ecosystem, section 4.1) reports the association of contractual coverage with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 7.9) its association with manual alteration.

4 Security Posture and Provider Concentration

Third-party security posture and provider concentration were associated with the wider score. They concern different exposures: the condition of participating services and dependence on a limited set of providers. The written base is smaller than the rating base. Security posture concerns the condition of participating services, while concentration concerns the resilience of dependence on a limited set of providers. Comment counts do not measure their relative importance.

4.1 Security Posture of Third-Party Tools and Sub-Processors

Indicator C180 · Rate the security posture of the third-party tools, platforms, and sub-processors the engagement's agents connect to · Full statistical analysis ›

Poor reported security posture of third-party tools and sub-processors was associated with low scores. The indicator extends governance to parties whose systems or services participate in the agent workflow. Its written evidence is small and cannot establish the extent of security exposure. The stronger quantitative base remains respondents' ratings, which should not be represented as an independent security assessment.

Of the 251 respondents who rated security posture 1 to 3, 192 had a score of 3 or below. Among the 462 who rated it 4 to 7, 74 (16.02%) did, and none of the 75 who rated it 8 to 10 did (chi-square = 308.25, p < 0.001, Cramér's V = 0.63).

Security posture was associated with the score computed without it at r = 0.84 (95% CI 0.78 to 0.88, n = 150 engagements) and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the partial r was −0.14 and was not significant (p = 0.083), the most negative of the ten. Respondents who rated it distressed recorded a score of 2.35 (n = 251), against 4.72 (n = 537) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.73, the strongest of the ten, and across the active stages at 0.69 to 0.76. After Holm adjustment, it was significant in all 86 segment levels tested. Within role it stood at 0.81 among individual contributors and 0.80 among engagement managers and portfolio directors. Among individual contributors alone, 76.67% (161 of 210) of those who rated it 1 to 3 had a score of 3 or below.

Nine respondents on 8 engagements wrote about security posture, and four of the answers were negative. Three of the four described one of the 26 forms of a control bypassed or not working, two of them problems raised with no response. The answers described a provider login shared by a whole team with no record of who used it, and a security review that stopped at the first layer of suppliers. An individual contributor added that the list of dependencies requested in the first month had never been written.

“Truthfully nobody sat down and wrote a list of what we depend on. I asked for it in month one and there is still no list. That is not really a third party fault though, they publish their notes, the problem is on our end that nobody reads them or tells us. The card scheme side actually gave notice about the sandbox refresh, it was in an email somewhere, only it went to a person who left. So I would not say the providers are bad. The security posture of the smaller tools we plugged in, that one I do worry about, we connected two or three things without anyone checking them properly and I raised it and got no answer. Also the licences, I do not think anyone thought about an agent making calls instead of a person, that came up when the quota looked strange and we had to discuss about it for days. But to be fair to everyone this account was only three months and half the access never came through, so there was never a moment where somebody could stop and map it all out. You cannot map what you have not been given. I think if we had another six months it would have been fine, or at least better… hard to say now.” -Individual Contributor, ITO, Technology engineering and operations

Part 2 (Third-Party Ecosystem, section 6.1) reports the association of security posture with the four study measures.

4.2 Management of Dependence on a Small Number of Model and Cloud Providers

Indicator C193 · Rate how well the engagement manages its dependence on a small number of model and cloud providers · Full statistical analysis ›

Weak management of dependence on a small number of model and cloud providers was associated with low scores. This concerns concentration of operational dependency, distinct from knowing which providers exist in the chain. The control question is whether that dependence is managed within the engagement's governance arrangements. The findings do not establish the probability or consequences of a provider disruption.

Of the 257 respondents who rated the management of concentration 1 to 3, 199 had a score of 3 or below, the third-highest share of the ten. Among the 474 who rated it 4 to 7, 67 (14.14%) did, the lowest such share of the ten, and none of the 71 who rated it 8 to 10 did (chi-square = 339.86, p < 0.001, Cramér's V = 0.65).

The indicator was associated with the score computed without it at r = 0.84 (95% CI 0.79 to 0.88, n = 150 engagements) and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the partial r was −0.04 (p = 0.646), the nearest to zero of the ten. Respondents who rated it distressed recorded a score of 2.32 (n = 257), against 4.78 (n = 545) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.72 and ranged across the active-stage groups from 0.73 in Build-Out to 0.77 in Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.81 among individual contributors and engagement managers and 0.80 among portfolio directors. Among individual contributors alone, 77.83% (165 of 212) of those who rated it 1 to 3 had a score of 3 or below.

Twenty-six respondents on 24 engagements wrote about provider concentration, and 19 of the answers were negative, 7 from canceled and 12 from active engagements. Four of the 19 described one of the 26 forms of a control bypassed or not working. Respondents described a single model provider with no second path, and a retirement notice that the engagement detected in time and could not act on.

“Only real worry is we are on one cloud provider for everything, if that region goes we have nothing, rest of the feeds have been steady enough” -Individual Contributor, ITO, Technology engineering and operations

“When our model provider retired a version this spring, we had thirty days' notice and the reclassification of the client's scanned document types had to be revalidated inside it. Nothing broke, but we have one provider and no second path if that window ever gets shorter.” -Portfolio Director, BPO, Corporate and back-office functions

Part 2 (Third-Party Ecosystem, section 6.2) reports the association of the management of concentration with the four study measures.

5 Fallback Capacity and Retention of Operational Knowledge

Converting informal human fixes into explicit checks was associated with the wider score. This connects operational knowledge with the visibility and repeatability of control after agent work is introduced. The separately coded habit pattern was not clearly associated with the rating. That result qualifies the measurement relationship without demonstrating that personal knowledge or fallback arrangements were adequate.

5.1 Replacement of Informal Human Fixes With Explicit Checks

Indicator C192 · Rate how deliberately the quiet human fixes for third-party quirks were replaced with explicit checks when agents took over the work · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.16 · Full statistical analysis ›

Failure to convert informal human fixes into explicit checks was associated with low scores. The indicator concerns whether knowledge previously carried by people becomes a visible and repeatable part of agent-era control. The separately coded habit pattern was not clearly associated with the rating. That comparison raises a measurement and interpretation question but does not formally demonstrate that the indicator cannot detect the practice.

Of the 267 respondents who rated the replacement of informal fixes 1 to 3, 201 had a score of 3 or below. Among the 459 who rated it 4 to 7, 68 (14.81%) did, and none of the 70 who rated it 8 to 10 did (chi-square = 315.03, p < 0.001, Cramér's V = 0.63).

The indicator was associated with the score computed without it at r = 0.85 (95% CI 0.80 to 0.89, n = 150 engagements) and with governance in the other four categories at r = 0.77. After adjustment for the mean of the other 53 indicators, the partial r was −0.11 (p = 0.183). Respondents who rated it distressed recorded a score of 2.37 (n = 267), against 4.76 (n = 529) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.72 and ranged across the active-stage groups from 0.70 in Build-Out to 0.81 in Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.82 among individual contributors, 0.80 among engagement managers, and 0.76 among portfolio directors. Among individual contributors alone, 78.34% (170 of 217) of those who rated it 1 to 3 had a score of 3 or below.

Habit in place of a control did not follow the rating. It was described by 10.11% (27 of 267) of respondents who rated the replacement of informal fixes 1 to 3, 10.24% (47 of 459) of those who rated it 4 to 7, and 14.29% (10 of 70) of those who rated it 8 to 10 (chi-square p = 0.567). Informal fixes persisted as habit on engagements where respondents judged them to have been replaced.

Nine respondents on 9 engagements wrote about informal fixes, and 7 of the answers were negative. Four of the 7 described one of the 26 forms of a control bypassed or not working, two of them habit in place of a control. Respondents described manual checks that had never been written down when the agents took over, and an engagement manager observed that payroll corrections had lived in one supervisor's memory for eleven years.

“directory sync and the ticketing api have been steady, no complaint from me. one thing not asked, the small manual checks we used to do on stale accounts is not written anywhere, so when agent take over that step nobody know it existed” -Individual Contributor, ITO, Technology engineering and operations

“The question about where people used to quietly fix things assumes somebody wrote it down. Nobody did. Payroll deduction corrections lived in one supervisors head for eleven years. You cant score that from one to ten. You can only find out after she leaves.” -Engagement Manager, BPO, Corporate and back-office functions

Part 2 (Third-Party Ecosystem, section 7.4) reports the association of the indicator with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 7.16) its association with manual alteration.

6 The Dependency Chain and Gated External Rails

Dependency mapping and management of gated external rails were associated with the wider score. Mapping concerns knowledge of the full chain, while gated rails concern constraints imposed by systems such as payment networks and regulatory submissions. The distinction matters for governance because knowing a dependency does not establish authority to change it. Conditional percentage rankings do not measure the comparative importance of those responsibilities.

6.1 Mapping of Third- and Fourth-Party Dependencies

Indicator C196 · Rate how completely the engagement has mapped every third and fourth party its agents depend on, down the full chain · Crossed with manual alteration in Part 5, Leading Dynamics of Manual Alteration, section 7.13 · Full statistical analysis ›

Incomplete mapping of third- and fourth-party dependencies was associated with low scores. Oversight requires visibility through the full chain of services on which agents depend. Mapping is a condition for understanding the chain, distinct from the ability to control every party within it. The adjusted comparison did not detect a clear separate association, and the conditional percentage ranking does not establish relative control importance.

Of the 252 respondents who rated dependency mapping 1 to 3, 185 had a score of 3 or below. Among the 464 who rated it 4 to 7, 79 (17.03%) did, the highest share at those ratings of the 47 governance indicators, and none of the 75 who rated it 8 to 10 did (chi-square = 275.01, p < 0.001, Cramér's V = 0.59).

Dependency mapping was associated with the score computed without it at r = 0.84 (95% CI 0.79 to 0.88, n = 150 engagements) and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the partial r was −0.13 (p = 0.105). Respondents who rated it distressed recorded a score of 2.43 (n = 252), against 4.72 (n = 539) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.66 and ranged across the active-stage groups from 0.71 in Build-Out to 0.78 in Production. After Holm adjustment, it was significant in 85 of the 87 segment levels tested. Within role it stood at 0.81 among individual contributors and 0.78 among engagement managers and portfolio directors. Among individual contributors alone, 76.59% (157 of 205) of those who rated it 1 to 3 had a score of 3 or below.

Forty-four respondents on 37 engagements wrote about dependency mapping, and 35 of the answers were negative, 21 from canceled and 14 from active engagements. Of the 35, 13 described one of the 26 forms of a control bypassed or not working, most often change discovered only through failure (7). Respondents described engagements that knew their direct providers and no list beyond the first tier, including engagements where contracts had been tightened and the map had not been drawn.

“Two months in, contracts with the market data providers have been reviewed and they now carry notice terms for interface changes, we did not have that at kickoff. What has not been done is the dependency map. We know our direct providers, we dont know who sits behind them, and nobody has looked at whether our licences cover machine users at all.” -Engagement Manager, ITO, Technology engineering and operations

“The full list of our dependencies has never been written down. I know our own test tools and the model behind them, but under that there are libraries and one data service the client environment calls, and I could not tell you who owns those. Last month a licence expired on a plug-in and the whole nightly run failed, nobody knew until morning.” -Individual Contributor, ITO, Technology engineering and operations

Part 2 (Third-Party Ecosystem, section 8.1) reports the association of dependency mapping with the four study measures, and Part 5 (Leading Dynamics of Manual Alteration, section 7.13) its association with manual alteration.

6.2 Management of Dependence on Gated External Rails

Indicator C197 · Rate how well the engagement manages agent dependence on gated external rails such as payment networks and regulatory submission systems · Full statistical analysis ›

Weak management of dependence on gated external rails was associated with low scores. Payment networks and regulatory submission systems can impose boundaries that the two firms cannot change through internal agent design. The governance question is how the engagement recognizes and manages those constraints. A large low-score share among low ratings is a descriptive association, not a validated measure of diagnostic reliability.

Of the 245 respondents who rated the management of gated rails 1 to 3, 197 had a score of 3 or below. Among the 471 who rated it 4 to 7, 71 (15.07%) did, and none of the 78 who rated it 8 to 10 did (chi-square = 351.73, p < 0.001, Cramér's V = 0.67).

The indicator was associated with the score computed without it at r = 0.84 (95% CI 0.78 to 0.88, n = 150 engagements) and with governance in the other four categories at r = 0.76. After adjustment for the mean of the other 53 indicators, the partial r was −0.08 (p = 0.333). Respondents who rated it distressed recorded a score of 2.27 (n = 245), against 4.76 (n = 549) among the rest (cluster-robust p < 0.001).

The association was observed within the canceled engagements at r = 0.69 and across the active stages at 0.79 in Build-Out, 0.72 in Transition/Cut-Over, and 0.78 in Production. After Holm adjustment, it was significant in 86 of the 87 segment levels tested. Within role it stood at 0.82 among individual contributors, 0.78 among engagement managers, and 0.71 among portfolio directors. Among individual contributors alone, 81.68% (165 of 202) of those who rated it 1 to 3 had a score of 3 or below, the highest such share of the 47.

Fourteen respondents on 12 engagements wrote 15 answers about gated rails, and 13 were negative, 8 from canceled and 5 from active engagements. Two of the 13 described problems raised with no response. Respondents described certification and filing portals built for a person holding a token, a matching utility that opened twice a day, and engagements where notice from the providers had improved while access to the rails had not.

“My low marks are really about matching utility… it opens for us twice a day only, and if we miss window there is nothing else, whole batch waits till tomorrow” -Individual Contributor, BPO, Trade, settlement and reconciliation

“Weakest point here is that some of what our agents must touch is still a screen designed for a person, particularly on the network side, and that is the one place I marked down. But I would not lay that at the provider's door as a failing. Card networks carry twenty years of member banks built around those screens, and an interface change there is not a release, it is a migration for several thousand institutions. They have given us honest notice of every change that mattered so far, in one case three months of it, which is more than we manage internally between our own teams. The fragility people speak about in this area is mostly ours, not theirs. We built agents on top of somebody else's roadmap and then treat the roadmap as a risk. I would rather we planned for their calendar than complained about it.” -Portfolio Director, BPO, Payments, cards and disputes

Part 2 (Third-Party Ecosystem, section 8.2) reports the association of the management of gated rails with the four study measures.