Source: https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/comparative-standing-of-the-five-failure-categories/
Author: Phil Hatch
Publisher: Akholi

Comparative Standing of the Five Failure Categories

The 2026 Report · Part 2

# Comparative Standing of the Five Failure Categories

Relative ratings, distressed answers, and shared associations across five categories

By Phil Hatch · Akholi · Version 1.8.2

Complete study · PDFPart 2 · PDFStudy overview / Part 2 / Comparative Standing of the Five Failure Categories

Respondents generally assessed AI technology and external systems more favorably than the organizational arrangements under which they operated. Table 2.1 reports category means. Section 1 compares the proportions of distressed ratings. The category labels organize the survey and do not establish sources of failure.

Table 2.1. The five failure categories ranked by mean rating

| Failure category | Indicators | Mean rating | Distressed answers | Distressed share | Correlation with cancellation, range across the indicators |
| --- | --- | --- | --- | --- | --- |
| Financial institution readiness | 20 | 3.55 | 8,219 of 15,966 | 51.48% | −0.60 to −0.56 |
| Co-management | 20 | 3.65 | 7,866 of 15,968 | 49.26% | −0.65 to −0.61 |
| Vendor capability | 20 | 3.67 | 7,687 of 15,943 | 48.22% | −0.60 to −0.56 |
| AI technology | 20 | 4.60 | 5,113 of 15,911 | 32.14% | −0.69 to −0.63 |
| Third-party ecosystem | 20 | 4.64 | 5,062 of 15,947 | 31.74% | −0.66 to −0.63 |

Note: The mean is the arithmetic average. Correlation describes the direction and strength of a linear association. Distressed-answer counts pool ratings across the indicators in a category and do not count distinct respondents.

Some written accounts attributed difficulties to preparation, access, or management arrangements while describing the technology itself more positively.

> “The tool was the least of it, in truth. My people spent their evenings re-keying metadata on deal documents the agent had already processed once, and kindly note that is what wore them down, not anything the model got wrong.” -Engagement Manager, BPO, Corporate and back-office functions

> “AI is not what fails here. It reads a long spec paper and finds the right section, that part is fine … I would ask you to understand nobody gave it clean test data either.” -Individual Contributor, ITO, Trade, settlement and reconciliation

> “To be fair the agent itself is steady, it gives me the same answer most times and the facts come out right. What moves underneath it is the manager Dashboard, they push a change and the flow breaks, then everyone says the AI is unreliable. It is not the AI, it is the platform below it” -Individual Contributor, CXM, Corporate and back-office functions

> “AI gets blamed for things that are not its fault. Half the flags it raises wrong are because the exposure file arrives before the month close is done.” -Individual Contributor, KPO, Credit, underwriting and suitability

> “the security approval queue is the real blocker here, not the model” -Individual Contributor, ITO, Technology engineering and operations

The category ranking describes relative ratings within this sample. It does not identify the cause of weak outcomes. Shared variance and the nonsignificant adjusted partial tests limit claims about the independent contribution of any category or indicator.

The written evidence differentiates the categories by the tasks and constraints respondents described. Detailed accounts are retained under the relevant indicators. Part 5 consolidates the measurement and disclosure of manual intervention.

> What I did is change the output before sending, because the AI give different answer for same case every time and only one of them can be correct, so somebody has to pick, and that somebody end up being me most of the time. And this is not one case, this is basically every case I touch, nine out of ten probably, I dont even bother counting exact anymore because the number stop meaning anything after a while. I dont want to say exactly who all knows about this on my side, its not really a straight answer, some people know some of it, nobody knows all of it probably including me. The informations coming in from HR system and payroll dont even match each other half the time, so before I even get to the AI part I already fixing their data, and then the AI part on top of that just add another layer I have to fix. Some case take me five minute extra, some take twenty, and there no way to plan for which kind you get until you already in it, and then the next case load and I start over again, same routine, and I have honestly stopped expecting it to change” INDIVIDUAL CONTRIBUTOR · BPO, Corporate and back-office functions

## 1 Concentration of Distress in Three Categories

Distressed ratings were more frequent in the three organizational categories. Financial institution readiness had 8,219 of 15,966 answers classified as distressed (51.48%). Co-management had 7,866 of 15,968 (49.26%), and vendor capability had 7,687 of 15,943 (48.22%). The corresponding shares were 5,113 of 15,911 (32.14%) for AI technology and 5,062 of 15,947 (31.74%) for the third-party ecosystem. These are pooled answer counts across each category's 20 indicators, not counts of distinct respondents.

Figure 2.1. Share of answers in the distressed range, by failure category

Scroll horizontally to view the complete figure.

Distressed ratings were more frequent in the three organizational categories.

Exact chart data

| Series | Point index | Category | Value as stored |
| --- | --- | --- | --- |
| Share of answers in the distressed range, by failure category | 0 | Financial institution readiness | 51.48 |
| Share of answers in the distressed range, by failure category | 1 | Co-management | 49.26 |
| Share of answers in the distressed range, by failure category | 2 | Vendor capability | 48.22 |
| Share of answers in the distressed range, by failure category | 3 | AI technology | 32.14 |
| Share of answers in the distressed range, by failure category | 4 | Third-party ecosystem | 31.74 |

Note: Distressed answers by category are financial institution readiness, 8,219 of 15,966. Co-management, 7,866 of 15,968. Vendor capability, 7,687 of 15,943. AI technology, 5,113 of 15,911. And third-party ecosystem, 5,062 of 15,947. A distressed rating is 3 or below on the favorable-direction scale. A respondent may contribute answers to multiple items.

Respondents on canceled engagements rated every category lower than those on active engagements. Individual contributors also rated each category lower than engagement managers and portfolio directors. The exact category ordering varied by group: co-management was lowest among canceled engagements. Table 2.2 reports the mean of the 20 indicator means within each group.

Table 2.2. Mean rating of each failure category by engagement status, lifecycle stage and respondent role

| Failure category | Canceled | Active, Build-Out | Active, Transition/Cut-Over | Active, Production |
| --- | --- | --- | --- | --- |
| Financial institution readiness | 2.36 | 4.14 | 4.27 | 4.32 |
| Co-management | 2.22 | 4.67 | 4.61 | 4.07 |
| Vendor capability | 2.46 | 4.41 | 4.29 | 4.60 |
| AI technology | 3.16 | 5.46 | 5.47 | 5.34 |
| Third-party ecosystem | 3.13 | 5.42 | 5.62 | 5.52 |

| Failure category | Individual contributors | Engagement managers | Portfolio directors |
| --- | --- | --- | --- |
| Financial institution readiness | 3.39 | 3.81 | 4.67 |
| Co-management | 3.52 | 3.90 | 4.98 |
| Vendor capability | 3.54 | 3.95 | 4.60 |
| AI technology | 4.44 | 4.93 | 5.46 |
| Third-party ecosystem | 4.48 | 4.94 | 5.96 |

Among active engagements, the co-management mean was 4.67 in Build-Out and 4.07 in Production, the largest observed stage difference among the categories. Portfolio directors rated each category more than one point above individual contributors. These descriptive contrasts do not establish whether stage or role independently accounts for the differences.

## 2 Uniform Association of Every Indicator With the Study Measures

Higher ratings on each of the 100 indicators were associated with lower cancellation, higher customer satisfaction among active engagements, a higher leave-one-out engagement indicator mean, and lower intent to leave. Each indicator passed its 22 applicable checks distributed across the four outcomes, for 2,200 checks in total. Passing these checks does not address every possible source of bias.

The associations also lay in a narrow band. The correlation coefficient r describes the direction and strength of a linear association, and n identifies the number of observations. Against cancellation, the 100 correlations ranged from r = −0.69 to r = −0.56 (n = 150 engagements each). Against customer satisfaction among active engagements, they ranged from r = 0.59 to r = 0.75 (n = 94 engagements each). Against the engagement indicator mean, they ranged from r = 0.76 to r = 0.88 (n = 150 engagements each). Against intent to leave, they ranged from r = −0.69 to r = −0.56.

No category stood outside the band. Each indicator of the two technology categories, rated highest, was associated with cancellation at r = −0.63 or stronger (n = 150 engagements). The three categories rated lowest ranged from r = −0.65 to r = −0.56 (n = 150 engagements each).

Low ratings tended to occur together. Respondents assessing data poorly also tended to assess management, vendor readiness, and technology poorly. This covariance is consistent with shared engagement conditions, common respondent perceptions, or both. It does not establish that operational failures occurred simultaneously or caused one another.

> Numbers shift between runs even though nothing changed on our end, mostly because their system keeps timing out mid query, so I adjust output manually before it gets sent, happens enough that I check every run now.” INDIVIDUAL CONTRIBUTOR · BPO, Corporate and back-office functions

← Part 2 overviewFinancial Institution Readiness →


## Links to related study evidence

- [Part 2](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/)
- [AI technology](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/ai-technology/)
- [Part 5](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-5-manual-alteration/)
- [Financial institution readiness](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/financial-institution-readiness/)
- [Co-management](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/co-management/)
- [vendor capability](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/vendor-capability/)
- [third-party ecosystem](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/third-party-ecosystem/)
- [customer satisfaction](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-1-measure-customer-satisfaction/)
- [engagement indicator mean](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-1-measure-engagement-indicator-mean/)
- [intent to leave](https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/study-data/part-4-c019/)