Source: https://akholi.com/outsourcing-2026-agentic-ai-tier-1-banks/part-2-five-failure-categories/vendor-capability/ Author: Phil Hatch Publisher: Akholi Vendor Capability The 2026 Report · Part 2 # Vendor Capability Leadership, management processes, knowledge, experience, talent, and delivery practices By Phil Hatch · Akholi · Version 1.8.2 Complete study · PDFPart 2 · PDFStudy overview / Part 2 / Vendor Capability Vendor capability was rated almost as low as the financial institution's own readiness. Its 20 indicators had a mean of 3.67 on the 10-point scale across the 150 engagements, and 7,687 of their 15,943 answers (48.22%) were distressed. The respondents rating the vendor were its own employees. They rated the vendor's leadership, commercial model, management processes, knowledge, production experience, talent, training, playbook, quality assurance, and platforms, and placed each one in the distressed range in about half of the answers. Table 2.5. The indicators of vendor capability, from the lowest mean rating | Indicator | Mean rating | Distressed answers | Distressed share | | --- | --- | --- | --- | | 3.2 Depth of Vendor Knowledge of Agentic AI | 3.58 | 416 of 807 | 51.55% | | 1.1 Effectiveness of Vendor Top Leadership | 3.60 | 381 of 792 | 48.11% | | 2.4 Vendor Qualification for Elite Agentic Outsourcing to Financial Institutions | 3.61 | 385 of 803 | 47.95% | | 2.3 Management of Agentic Engagements for Global Financial Institutions | 3.63 | 381 of 779 | 48.91% | | 4.1 Vendor Experience Running Agents Beyond Pilots | 3.64 | 383 of 794 | 48.24% | | 3.1 Depth of Vendor Knowledge of the Financial Industry | 3.64 | 392 of 802 | 48.88% | | 1.3 Pricing of Agentic Work When Hours No Longer Measure Value | 3.65 | 383 of 795 | 48.18% | | 6.1 Immaturity of the Playbook From Concept to Governed Production | 3.65 | 392 of 807 | 48.57% | | 5.3 Adequacy of Vendor Investment in Training | 3.66 | 388 of 789 | 49.18% | | 2.1 Vendor Management Processes Left Unchanged From Staffed Delivery | 3.66 | 388 of 799 | 48.56% | | 7.1 Accountability for Agent Error Left With the Analyst on Shift | 3.67 | 373 of 794 | 46.98% | | 5.2 Process and Leadership Development Programs | 3.67 | 393 of 803 | 48.94% | | 6.2 Quality Assurance for Non-Deterministic Agent Output | 3.68 | 377 of 794 | 47.48% | | 1.2 Transformation of the Vendor Commercial Model | 3.69 | 386 of 812 | 47.54% | | 2.2 End-to-End Management of Agentic Engagements | 3.69 | 364 of 787 | 46.25% | | 4.2 Operation of Large Numbers of Agents Without an Operations Function | 3.69 | 379 of 798 | 47.49% | | 5.4 Practical Agentic Skills Built by Vendor Training | 3.69 | 380 of 804 | 47.26% | | 5.1 Supply of Qualified Agentic Talent | 3.69 | 382 of 802 | 47.63% | | 6.3 Optimization of Vendor Platforms for Agentic Outsourcing | 3.70 | 384 of 794 | 48.36% | | 5.5 Financial Industry Knowledge Built by Vendor Training | 3.72 | 380 of 788 | 48.22% | Mean ratings in Table 2.5 are computed across respondents. The category figure of 3.67 is computed across engagements. Section 8 reports one further question on the vendor, its preparedness compared with five years earlier. The survey placed that question in its section on the respondent's relationship with the vendor, and it is not one of the category's 20 indicators. Negative vendor accounts described limited production experience, delivery methods adapted from staffed outsourcing, general training, and scarce experienced personnel shared across engagements. Some respondents reported learning through live delivery. These accounts identify perceived gaps without establishing that all vendors lacked adequate methods or that commercial commitments systematically exceeded capability. > “We are sold into a capital-markets client as though the firm has done investor communications with agents before, and it has not. There is one person in the whole delivery organisation who understands what a quiet period means, and she is not on this engagement … Internally the machinery has not changed at all, same weekly status pack, same utilisation questions, none of which describe what my team actually does now.” -Engagement Manager, CXM, Client servicing and communications > “What I would want noted above everything else is that our leadership is good at winning this kind of work and we are still learning how to run it. Nobody at home has answered who pays when an agent gets an onboarding step wrong.” -Engagement Manager, CXM, Corporate and back-office functions > “They sold this in about eight weeks and then handed us a delivery model built for staffed work - that gap is most of the story … What is missing is anyone senior owning the space between a signed pilot and a working operating rhythm.” -Engagement Manager, KPO, Corporate and back-office functions The vendor-comment question received answers from 530 of 850 respondents (62.35%). Some distinguished capable colleagues and immediate managers from deficiencies they attributed to senior support, preparation, or commercial arrangements. Favorable accounts of training and operational support are retained in the relevant subsections. > “My own lead is very good, that is not what these numbers say. The gap is higher up, whoever writes the training deck has not run an agent on a live pipeline, we get slides about concepts and then work the rest out on the platform ourselves. So the low marks are about the company, not my team” -Individual Contributor, ITO, Technology engineering and operations The category's correlations occupied a narrow range: −0.60 to −0.56 for cancellation and 0.85 to 0.88 for the engagement indicator mean (n = 150 engagements for each estimate). These coefficients do not establish a ranking of independent importance. ## 1 Leadership and the Commercial Model Vendor leadership and commercial arrangements were reported to retain gaps in accountability and tension between agentic delivery and staffing-based revenue. Three indicators describe the vendor at the level of the firm. They cover the effectiveness of its top leadership and the transformation of its commercial model as agents reduce the billable human effort it sells. The third is its ability to price agentic work profitably once hours no longer measure value. The effectiveness of leadership was distressed in 381 of 792 answers (48.11%), the transformation of the commercial model in 386 of 812 (47.54%), and the pricing of agentic work in 383 of 795 (48.18%). All three were associated with each core study measure, and no association failed any of the applicable robustness checks. Leadership comments described perceived gaps in senior support, decision authority, and experience with agentic delivery. Some respondents praised immediate managers while criticizing the assistance available to them from more senior personnel. The commercial items assess different constructs: adaptation of the business model and ability to price agentic work profitably. Both attracted predominantly negative comments. Part 1, section 4.1 of “Structural Attributes and Outcome Associations,” describes actual reported pricing arrangements. The commercial ratings concern respondents' assessments of the vendor's response to those arrangements. ### 1.1 Effectiveness of Vendor Top Leadership Indicator C115 · Rate the overall effectiveness of your employer’s top leadership team · Full statistical analysis › Respondents described gaps in vendor leadership accountability and experience, including uncertainty about who owned decisions and remediation across the engagement. The mean rating was 3.6 and the median 4.0 (n = 792 respondents). The mean rating of leadership on canceled engagements, 2.4 (n = 304 respondents), was lower than on active engagements in any stage. Figure 2.44. Effectiveness of Vendor Top Leadership: associations with the study measures Scroll horizontally to view the complete figure. Effectiveness of vendor top leadership: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.58 [−0.68, −0.46] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.71 [0.59, 0.79] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.89] 150 engagements p < 0.001 | | Intent to leave | | r = −0.60 [−0.67, −0.52] 741 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.67780934609999999 | 1 | | 95% confidence interval | 1 | -0.46348462400000001 | 1 | | Reported association | 0 | -0.58061852970000005 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.58701022680000003 | 1 | | 95% confidence interval | 1 | 0.79468226070000003 | 1 | | Reported association | 0 | 0.70569328129999997 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80682250870000005 | 1 | | 95% confidence interval | 1 | 0.8939326125 | 1 | | Reported association | 0 | 0.85635508959999995 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66589466980000001 | 1 | | 95% confidence interval | 1 | -0.52227429049999996 | 1 | | Reported association | 0 | -0.59823141670000002 | 1 | Note: Survey question: “Rate the overall effectiveness of your employer’s top leadership team”. Distressed ratings: 381 of 792 answers (48.11%). Group means and numeric-answer bases: Individual contributor (n = 609): 3.46. Engagement manager (n = 150): 3.99. Portfolio director (n = 31): 4.52. Build-Out (n = 164): 4.27. Transition/Cut-Over (n = 229): 4.35. Production (n = 95): 4.46. Canceled (n = 304): 2.42. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The effectiveness of the vendor's top leadership was associated with all four core study measures, and each association held under the applicable robustness checks. Engagements whose respondents rated leadership more effective had a lower cancellation rate (r = −0.58, 95% CI −0.68 to −0.46, n = 150 engagements). Customer satisfaction among active engagements was higher where leadership was rated more effective (r = 0.71, 95% CI 0.59 to 0.79, n = 94 engagements). The engagement indicator mean, computed without the leadership indicator, was higher too (r = 0.86, 95% CI 0.81 to 0.89, n = 150 engagements). Respondents who rated their leadership higher reported lower intent to leave (r = −0.60, cluster bootstrap 95% CI −0.67 to −0.52, n = 741 respondents). Distressed ratings came from 381 of the 792 answers (48.11%), and the engagement mean was distressed on 60 of the 150 engagements. The written responses addressing leadership numbered 30, and 23 were negative. Respondents placed the weakness at levels senior to their immediate managers. They described the absence of anyone who owned the engagement end to end or who would answer for an agent's error, and senior people who had not run agentic work before. Accounts of training were mixed. > “Training was actually useful, we got proper sessions on the matching Agent and i could use it directly on custodian breaks... that part improved. Top management is different matter. nobody owns it end to end, and when exception queue builds up nobody is watching.” -Individual Contributor, BPO, Trade, settlement and reconciliation > “What no question here asks is who I am supposed to ring at two in the morning when the agent has miscalculated a run and the file is already sitting with their bank. There is a playbook for a staffed process, there is nothing for this, I asked twice and got a name that turned out to be somebody who left … it is that nobody above has thought past winning the work.” -Engagement Manager, BPO, Corporate and back-office functions > “What hasnt changed is anyone above my manager. Nobody at the top has run one of these to the end, and it shows the minute a client asks a hard question.” -Individual Contributor, ITO, Regulatory, audit, tax and legal > “My reservation sits higher up. Nobody has told me who signs the letter if one of these responses goes out wrong.” -Portfolio Director, CXM, Corporate and back-office functions > “and leadership send mails about ai every month but nothing reach the floor, no training, no time set aside. we are learning the policy assistant on live queries and hoping we catch the wrong ones” -Individual Contributor, CXM, Corporate and back-office functions Negative leadership comments and more negative engagement-level sentiment were associated with higher cancellation. The accounts concerned responsibility for decisions and remediation. They do not establish that senior leadership conduct caused cancellation. ### 1.2 Transformation of the Vendor Commercial Model Indicator C116 · Rate how well your employer is transforming its commercial model as agentic AI reduces the billable human effort it sells today · Full statistical analysis › Accounts of vendor commercial-model change identified tension between agentic delivery and revenue or internal targets based on billable human effort. The mean rating was 3.7 and the median 4.0 (n = 812 respondents). The transformation of the commercial model was weakest on canceled engagements, where the mean rating was 2.4 (n = 306 respondents), lower than on active engagements in any stage. Figure 2.45. Transformation of the Vendor Commercial Model: associations with the study measures Scroll horizontally to view the complete figure. Transformation of the vendor commercial model: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.60 [−0.69, −0.48] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.68 [0.55, 0.77] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.87 [0.82, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.62 [−0.68, −0.54] 762 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.69016866340000005 | 1 | | 95% confidence interval | 1 | -0.4815248768 | 1 | | Reported association | 0 | -0.5958079557 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.54937262369999995 | 1 | | 95% confidence interval | 1 | 0.77326680390000002 | 1 | | Reported association | 0 | 0.6766681578 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.82373763590000004 | 1 | | 95% confidence interval | 1 | 0.90362699420000003 | 1 | | Reported association | 0 | 0.86924688169999997 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68223444509999998 | 1 | | 95% confidence interval | 1 | -0.5440691943 | 1 | | Reported association | 0 | -0.61750082709999998 | 1 | Note: Survey question: “Rate how well your employer is transforming its commercial model as agentic AI reduces the billable human effort it sells today”. Distressed ratings: 386 of 812 answers (47.54%). Group means and numeric-answer bases: Individual contributor (n = 629): 3.58. Engagement manager (n = 150): 3.91. Portfolio director (n = 31): 4.58. Build-Out (n = 166): 4.54. Transition/Cut-Over (n = 242): 4.25. Production (n = 98): 4.74. Canceled (n = 306): 2.44. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. All four core study measures were associated with the transformation of the vendor's commercial model, and none of the applicable robustness checks overturned an association. Cancellation was less common where the transformation was rated further advanced (r = −0.60, 95% CI −0.69 to −0.48, n = 150 engagements). Among active engagements, customer satisfaction was higher where the commercial model had been transformed (r = 0.68, 95% CI 0.55 to 0.77, n = 94 engagements). The engagement indicator mean computed without the commercial-model indicator moved closely with the rating (r = 0.87, 95% CI 0.82 to 0.90, n = 150 engagements). Respondents who rated the transformation higher reported lower intent to leave (r = −0.62, cluster bootstrap 95% CI −0.68 to −0.54, n = 762 respondents). Distressed ratings came from 386 of the 812 answers (47.54%), and the engagement mean was distressed on 56 of the 150 engagements. Commercial-model comments numbered 145. 132 were negative. Responses in the challenges field emphasized tension between agentic work and revenue based on human staffing. Longer comments on the vendor described reduced billable effort under particular arrangements and internal targets that continued to emphasize headcount. > “When the agents absorb most of the query volume my headcount goes down and the invoice goes down with it, and nobody has said what replaces it. The answer each time is that a new model is coming. Until it arrives we are, in effect, penalised for the very thing we were asked to build.” -Engagement Manager, CXM, Client servicing and communications > “The heavy work is the first six weeks of teaching the routing rules, and after that the volume costs us almost nothing, but we are still billing as though a person sits behind every ticket. Nobody above me has told us how the next phase is meant to be priced … ” -Engagement Manager, CXM, Technology engineering and operations > “vendor's own FTE model fights the agent's success” -Individual Contributor, CXM, Technology engineering and operations > “Vendor defended FTE margins over fixing the black box” -Engagement Manager, BPO, Trade, settlement and reconciliation No significant association was detected between commercial-model sentiment and the study measures. The comments describe a perceived incentive conflict under staffing-based arrangements, not a measured relationship between vendor revenue and automation gains. The quantitative outcome associations derive from the ratings. ### 1.3 Pricing of Agentic Work When Hours No Longer Measure Value Indicator C128 · Rate your employer’s ability to price agentic work profitably when billable hours no longer measure the value delivered · Full statistical analysis › Respondents described pricing that did not consistently reflect agentic delivery effort or support an agreed basis for profitable work. The mean rating was 3.6 and the median 4.0 (n = 795 respondents). Pricing received a mean rating of 2.5 on canceled engagements (n = 303 respondents), lower than on active engagements in any stage. Figure 2.46. Pricing of Agentic Work When Hours No Longer Measure Value: associations with the study measures Scroll horizontally to view the complete figure. Pricing of agentic work when hours no longer measure value: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.72 [0.60, 0.80] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.87 [0.82, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.60 [−0.67, −0.53] 748 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.65813792910000002 | 1 | | 95% confidence interval | 1 | -0.4351331001 | 1 | | Reported association | 0 | -0.55658137159999999 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.59946541080000004 | 1 | | 95% confidence interval | 1 | 0.80165710639999999 | 1 | | Reported association | 0 | 0.7152114598 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.82111341429999996 | 1 | | 95% confidence interval | 1 | 0.90212834789999996 | 1 | | Reported association | 0 | 0.86725089040000003 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66929463010000001 | 1 | | 95% confidence interval | 1 | -0.52652651029999997 | 1 | | Reported association | 0 | -0.60209021689999997 | 1 | Note: Survey question: “Rate your employer’s ability to price agentic work profitably when billable hours no longer measure the value delivered”. Distressed ratings: 383 of 795 answers (48.18%). Group means and numeric-answer bases: Individual contributor (n = 615): 3.54. Engagement manager (n = 150): 3.85. Portfolio director (n = 29): 4.72. Build-Out (n = 160): 4.34. Transition/Cut-Over (n = 235): 4.27. Production (n = 97): 4.59. Canceled (n = 303): 2.50. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The vendor's ability to price agentic work was associated with cancellation, customer satisfaction, the engagement indicator mean, and intent to leave, and each association withstood the applicable robustness checks. Engagements whose vendor priced agentic work better had a lower cancellation rate (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Customer satisfaction among active engagements was higher where pricing was rated better (r = 0.72, 95% CI 0.60 to 0.80, n = 94 engagements). Pricing was also closely associated with the engagement indicator mean computed without it (r = 0.87, 95% CI 0.82 to 0.90, n = 150 engagements). Intent to leave was lower among respondents who rated pricing higher (r = −0.60, cluster bootstrap 95% CI −0.67 to −0.53, n = 748 respondents). Distressed ratings came from 383 of the 795 answers (48.18%), and the engagement mean was distressed on 62 of the 150 engagements. Of the 137 written responses addressing pricing, 136 were negative. Respondents described uncertain prices, continuing disputes over commercial terms, and charges that did not reflect delivery effort. Longer accounts described headcount-based pricing and financial institution reports that, according to respondents, omitted the associated reduction in vendor margins. > “Put it plainly, we were sold as an agentic engagement and then priced and staffed like an ordinary ITO ramp. Nobody above me could explain what we bill for once the agent clears an access request itself.” -Engagement Manager, ITO, Technology engineering and operations > “We priced this one on a blended rate against estimated human effort, and then the entire premise of the engagement was that the human effort would fall, which meant every week we succeeded technically we lost money and every week we struggled we looked incompetent.” -Portfolio Director, ITO, Regulatory, audit, tax and legal > “The vendor priced and reported deliverable turnaround in hours claimed by the agent, without a separate line for the senior banker hours spent re-deriving key assumptions the agent could not justify, masking the true cost of the engagement.” -Individual Contributor, KPO, Corporate and back-office functions > “We are sold as tax research capability by people who have never priced work that is not measured in hours, and effect on me personally is that I defend a delivery model I did not design and do not believe in.” -Engagement Manager, KPO, Regulatory, audit, tax and legal > “Pricing conversations stall because nobody wants to admit FTE hours are dropping” -Engagement Manager, CXM, Technology engineering and operations More negative engagement-level pricing sentiment was associated with cancellation. More negative respondent-level sentiment was associated with higher intent to leave. The accounts suggest a mismatch between existing pricing methods and agentic delivery, although the study did not independently examine contracts, margins, or financial institution reports. > Training is the one I keep coming back to. When I joined this account I got two half days on the platform, mostly someone sharing their screen, and then a set of e-learning modules written for a different kind of work altogether. Nothing about HR data. Nothing about what an attrition table or a comp table should look like before you trust it. The person running the session had never opened a payroll extract in her life, that was obvious in ten minutes. The e-learning is a seperate world from what we actually do. Since then I have had nothing. My lead is good, he will sit with you if you ask, but he is only teaching me what he taught himself, and he learned it the same way I did, on live work. There is a certification the company talks about, I did most of it on my own weekends and the licence ran out before I finished. I am not saying the people here are not capable. We do the work. But it is all picked up sideways from whoever is sitting next to you, and when someone leaves it goes with them. Nine months on this account and there is still no written handbook for any of it. That is why those questions got a 1 from me.” INDIVIDUAL CONTRIBUTOR · KPO, Corporate and back-office functions ## 2 Management Processes and End-to-End Engagement Management Accounts of vendor management identified procedures inherited from staffed delivery, limited preparation for agentic work, and unclear end-to-end ownership. Four indicators assess the vendor’s management processes, end-to-end engagement management, capability with global financial institutions, and overall delivery qualifications. Quantitative ratings and written accounts address different aspects of these capabilities. Across the limited written evidence, respondents described management practices developed for staffed delivery, incomplete preparation for agentic work, and unclear engagement ownership. No written response specifically addressed capability with global financial institutions. ### 2.1 Adaptation of Vendor Management Processes Indicator C117 · Rate how well your employer’s internal management processes have been redesigned for agentic AI delivery rather than traditional staffed delivery · Full statistical analysis › Vendor management processes were reported to retain rosters, headcount reporting, and utilization targets developed for staffed delivery. The mean rating was 3.7 and the median 4.0 (n = 799 respondents). On canceled engagements, the redesign of management processes was rated lower than on active engagements in any stage (mean 2.5, n = 295 respondents). Figure 2.47. Adaptation of Vendor Management Processes: associations with the study measures Scroll horizontally to view the complete figure. Adaptation of vendor management processes: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.57 [−0.67, −0.45] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.68 [0.55, 0.78] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 | | Intent to leave | | r = −0.57 [−0.64, −0.49] 749 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.6664179603 | 1 | | 95% confidence interval | 1 | -0.44701307979999999 | 1 | | Reported association | 0 | -0.56667833749999996 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.55428059470000002 | 1 | | 95% confidence interval | 1 | 0.77608865920000003 | 1 | | Reported association | 0 | 0.68047562080000001 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80445060420000003 | 1 | | 95% confidence interval | 1 | 0.89256668159999997 | 1 | | Reported association | 0 | 0.85454240319999997 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.64321271609999997 | 1 | | 95% confidence interval | 1 | -0.48958077249999998 | 1 | | Reported association | 0 | -0.57069330910000005 | 1 | Note: Survey question: “Rate how well your employer’s internal management processes have been redesigned for agentic AI delivery rather than traditional staffed delivery”. Distressed ratings: 388 of 799 answers (48.56%). Group means and numeric-answer bases: Individual contributor (n = 616): 3.51. Engagement manager (n = 150): 4.03. Portfolio director (n = 31): 4.68. Build-Out (n = 171): 4.37. Transition/Cut-Over (n = 234): 4.30. Production (n = 99): 4.43. Canceled (n = 295): 2.48. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. Each of the four core study measures was associated with how far the vendor had redesigned its internal management processes, and the applicable robustness checks upheld every association. Engagements whose vendor had redesigned its processes further had a lower observed cancellation rate (r = −0.57, 95% CI −0.67 to −0.45, n = 150 engagements). Customer satisfaction among active engagements was higher where the processes had been redesigned (r = 0.68, 95% CI 0.55 to 0.78, n = 94 engagements). The engagement indicator mean computed without the process indicator was closely associated with the redesign (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated the redesign higher reported lower intent to leave (r = −0.57, cluster bootstrap 95% CI −0.64 to −0.49, n = 749 respondents). Distressed ratings came from 388 of the 799 answers (48.56%), and the engagement mean was distressed on 61 of the 150 engagements. Of the 25 written responses addressing vendor management processes, 23 were negative. Nearly all appeared in comments about the vendor. Respondents described continued reliance on shift rosters, monthly headcount reports, utilization targets, and checklists developed for manual work. > “My utilization target counts heads on the account, not tickets the agent closed, so every hour I spend tuning intent routing shows up as idle time on my own report. Nobody above me has said that out loud yet.” -Engagement Manager, CXM, Corporate and back-office functions > “our internal process is still built for staffed delivery, headcount reports, utilization, all of it, none of that tells anybody how model is performing. Third, stop putting people on production work whose only experience is pilot.” -Engagement Manager, CXM, Regulatory, audit, tax and legal > “Our internal process still assumes headcount. Every week I still fill the same shift Tracker even though the Agent is doing the routing... What needs to change is the reporting we do, it should measure tickets closed correctly, not hours logged.” -Individual Contributor, CXM, Technology engineering and operations > “My manager is good and I dont have complaints there. What the numbers dont show is that were still run like a body shop - hours booked, tickets closed - when half the work now is checking what the tool did.” -Individual Contributor, ITO, Client servicing and communications > “Nobody above me has redesigned the delivery playbook, they took the old seat-based model and put agents where the seats used to be, and the reporting rolls up exactly as it always did.” -Engagement Manager, CXM, Corporate and back-office functions Written sentiment on management processes showed no statistically significant association with the study measures. The 25 responses describe staff adapting established procedures to agentic delivery, sometimes by working outside them. This small qualitative sample provides context rather than a reliable estimate of the prevalence of those practices. ### 2.2 End-to-End Management of Agentic Engagements Indicator C119 · Rate your employer’s ability to manage agentic AI outsourcing engagements end to end · Full statistical analysis › Accounts of end-to-end vendor management contrasted inadequate preparation and unassigned engagement ownership with effective delivery that retained specific weaknesses. The mean rating was 3.7 and the median 4.0 (n = 787 respondents). Respondents on canceled engagements gave end-to-end management a mean rating of 2.5 (n = 294 respondents), lower than respondents on active engagements in any stage. Figure 2.48. End-to-End Management of Agentic Engagements: associations with the study measures Scroll horizontally to view the complete figure. End-to-end management of agentic engagements: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.72 [0.61, 0.81] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.61 [−0.68, −0.53] 738 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.65843254259999995 | 1 | | 95% confidence interval | 1 | -0.43555447600000002 | 1 | | Reported association | 0 | -0.55694011990000003 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.6056574414 | 1 | | 95% confidence interval | 1 | 0.80510424790000001 | 1 | | Reported association | 0 | 0.71992744900000005 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.81135272270000003 | 1 | | 95% confidence interval | 1 | 0.89653698049999997 | 1 | | Reported association | 0 | 0.85981383730000005 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.67782721879999996 | 1 | | 95% confidence interval | 1 | -0.53358145099999998 | 1 | | Reported association | 0 | -0.61021242170000001 | 1 | Note: Survey question: “Rate your employer’s ability to manage agentic AI outsourcing engagements end to end”. Distressed ratings: 364 of 787 answers (46.25%). Group means and numeric-answer bases: Individual contributor (n = 606): 3.58. Engagement manager (n = 150): 3.97. Portfolio director (n = 29): 4.38. Build-Out (n = 167): 4.35. Transition/Cut-Over (n = 232): 4.35. Production (n = 94): 4.55. Canceled (n = 294): 2.51. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The applicable robustness checks left intact each association between end-to-end management of agentic engagements and the four core study measures. Cancellation was less common on engagements with stronger end-to-end management (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Customer satisfaction among active engagements was higher where end-to-end management was rated stronger (r = 0.72, 95% CI 0.61 to 0.81, n = 94 engagements). The engagement indicator mean computed without the end-to-end indicator was closely associated with it (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Respondents who rated end-to-end management higher reported lower intent to leave (r = −0.61, cluster bootstrap 95% CI −0.68 to −0.53, n = 738 respondents). Distressed ratings came from 364 of the 787 answers (46.25%), the lowest distressed share of the 20 indicators of vendor capability, and the engagement mean was distressed on 65 of the 150 engagements. Of the 17 written responses addressing end-to-end management, 11 were negative, and 6 were positive. Nearly all appeared in comments about the vendor. Negative accounts described inadequate preparation and the absence of an accountable engagement owner. Positive accounts described generally effective delivery with specific remaining weaknesses. > “We were placed in front of a regulatory client with nobody who had run agents through a live filing cycle, and when it broke, ownership landed on my team and stayed there.” -Engagement Manager, KPO, Regulatory, audit, tax and legal > “Nobody put us on this account with a plan and not one question here reaches that. My lead did everything right, she was the only one who answered anything.” -Individual Contributor, CXM, Technology engineering and operations > “My employer was given three weeks to stand up the screening queue, kindly note that is not enough for KYC. The people tried. The preparation was not there” -Individual Contributor, BPO, Financial crime and client onboarding > “What has to change is above that. Somebody needs to own the whole thing end to end insted of four leads each owning a piece.” -Individual Contributor, ITO, Regulatory, audit, tax and legal > “End to end we run engagement well, I stand by that. But I have asked twice up the line who carries the cost if it goes wrong, and both times I got a workshop invitation.” -Engagement Manager, ITO, Technology engineering and operations More negative written sentiment was associated with cancellation and higher intent to leave. Because the qualitative analysis draws on only 17 responses, these associations should be treated as exploratory. The larger structured-rating sample provides the principal quantitative evidence. ### 2.3 Management of Agentic Engagements for Global Financial Institutions Indicator C121 · Rate your employer’s ability to manage agentic AI outsourcing engagements for global banks in particular · Full statistical analysis › Vendor capability to manage agentic AI outsourcing engagements for global financial institutions received lower ratings on canceled engagements than on active engagements in any measured stage. The mean rating was 3.6 and the median 4.0 (n = 779 respondents). This ability was weakest on canceled engagements, where the mean rating was 2.4 (n = 292 respondents), lower than on active engagements in any stage. Figure 2.49. Management of Agentic Engagements for Global Financial Institutions: associations with the study measures Scroll horizontally to view the complete figure. Management of agentic engagements for global financial institutions: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.70 [0.58, 0.79] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.59 [−0.66, −0.50] 732 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.65967506239999996 | 1 | | 95% confidence interval | 1 | -0.43733268619999999 | 1 | | Reported association | 0 | -0.55845354189999996 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.57551205620000001 | 1 | | 95% confidence interval | 1 | 0.78819428970000005 | 1 | | Reported association | 0 | 0.69686825470000002 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80873436710000002 | 1 | | 95% confidence interval | 1 | 0.89503243580000003 | 1 | | Reported association | 0 | 0.85781530589999999 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66449572430000003 | 1 | | 95% confidence interval | 1 | -0.50450824839999997 | 1 | | Reported association | 0 | -0.58946435990000001 | 1 | Note: Survey question: “Rate your employer’s ability to manage agentic AI outsourcing engagements for global banks in particular”. Distressed ratings: 381 of 779 answers (48.91%). Group means and numeric-answer bases: Individual contributor (n = 597): 3.50. Engagement manager (n = 150): 3.90. Portfolio director (n = 30): 4.50. Build-Out (n = 161): 4.39. Transition/Cut-Over (n = 235): 4.20. Production (n = 91): 4.64. Canceled (n = 292): 2.43. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. Each association between the management of engagements for global financial institutions and the four core study measures held in all applicable robustness checks. Engagements rated more capable on this measure had a lower cancellation rate (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Among active engagements, customer satisfaction was higher where this ability was rated higher (r = 0.70, 95% CI 0.58 to 0.79, n = 94 engagements). The engagement indicator mean, computed without this indicator, was higher too (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Intent to leave was lower among respondents who rated this ability higher (r = −0.59, cluster bootstrap 95% CI −0.66 to −0.50, n = 732 respondents). Distressed ratings came from 381 of the 779 answers (48.91%), and the engagement mean was distressed on 63 of the 150 engagements. No written response specifically addressed the vendor’s capability with global financial institutions. Consequently, this indicator supports conclusions about respondents’ ratings but provides no direct qualitative explanation for them. Sections 2.1, 2.2, and 2.4 examine related management capabilities. ### 2.4 Overall Vendor Qualifications for Agentic Outsourcing Indicator C134 · Rate your employer’s overall qualification to deliver elite agentic AI outsourcing services to banks · Full statistical analysis › Respondents described a gap between services sold as highly qualified agentic AI outsourcing and the production experience available during delivery. The survey described the service standard as “elite.” The mean rating was 3.6 and the median 4.0 (n = 803 respondents). The survey described the service standard as “elite.” The mean rating of the vendor's qualification on canceled engagements, 2.5 (n = 306 respondents), was lower than on active engagements in any stage. Figure 2.50. Overall Vendor Qualifications for Agentic Outsourcing: associations with the study measures Scroll horizontally to view the complete figure. Overall vendor qualifications for agentic outsourcing: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.72 [0.61, 0.81] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.58 [−0.65, −0.50] 757 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.65844438940000005 | 1 | | 95% confidence interval | 1 | -0.43557142209999999 | 1 | | Reported association | 0 | -0.55695454639999997 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.60693749850000001 | 1 | | 95% confidence interval | 1 | 0.80581518699999999 | 1 | | Reported association | 0 | 0.7209010548 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.81166687380000002 | 1 | | 95% confidence interval | 1 | 0.89671736430000004 | 1 | | Reported association | 0 | 0.86005352199999996 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.65268165489999996 | 1 | | 95% confidence interval | 1 | -0.49707240580000001 | 1 | | Reported association | 0 | -0.57920758289999996 | 1 | Note: Survey question: “Rate your employer’s overall qualification to deliver elite agentic AI outsourcing services to banks”. Distressed ratings: 385 of 803 answers (47.95%). Group means and numeric-answer bases: Individual contributor (n = 621): 3.46. Engagement manager (n = 150): 4.00. Portfolio director (n = 30): 4.63. Build-Out (n = 166): 4.33. Transition/Cut-Over (n = 234): 4.21. Production (n = 97): 4.49. Canceled (n = 306): 2.49. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The vendor's overall self-assessed qualification for agentic outsourcing showed an association with each of the four core study measures that survived all applicable robustness checks. Cancellation was less common where respondents rated the vendor better qualified (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Active engagements whose vendor was rated better qualified reported higher customer satisfaction (r = 0.72, 95% CI 0.61 to 0.81, n = 94 engagements). The engagement indicator mean, computed without the qualification indicator, was closely associated with the rating (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Respondents who judged the vendor better qualified reported lower intent to leave (r = −0.58, cluster bootstrap 95% CI −0.65 to −0.50, n = 757 respondents). Distressed ratings came from 385 of the 803 answers (47.95%), and the engagement mean was distressed on 67 of the 150 engagements. Of the 28 written responses addressing vendor qualifications, 17 were negative. Nearly all appeared in comments about the vendor. Negative accounts described services sold before the firm had acquired relevant production experience, with teams learning during live delivery. Several positive accounts also identified specific weaknesses. > “nobody in our firm has run early-warning work with agents before, so we are learning on a live portfolio, and the two seniors who do understand it are shared across other work. I do not think this is dishonesty, it is a firm selling something it has not built yet, which is a different fault.” -Engagement Manager, KPO, Credit, underwriting and suitability > “The problem is nobody above the trainer has done this before either. When the agent closed a batch of alerts wrong in March, my lead asked me what the process is, and i asked him the same thing. There is no process. So we made one up on the call and wrote it after.” -Individual Contributor, ITO, Financial crime and client onboarding > “We were staffed for a document backlog, not for standing up an agent platform, and please understand that nobody at our end ever pretended otherwise. The people they gave me worked hard. The company simply sold what it has yet to build.” -Engagement Manager, BPO, Corporate and back-office functions > “there is no box for saying the people here are fine and the firm above them is not.” -Individual Contributor, KPO, Corporate and back-office functions More negative written sentiment was associated with cancellation and higher intent to leave. These associations are exploratory because the qualitative sample contains only 28 responses. The accounts suggest a perceived gap between commercial commitments and delivery experience. They do not independently establish the firm’s qualifications. ## 3 Knowledge of the Financial Industry and of Agentic AI Financial-industry knowledge and agentic AI knowledge were reported as distinct capabilities, with strength in one not necessarily accompanied by strength in the other. Two indicators assess the depth of vendor knowledge in the financial industry and agentic AI. The distinction matters because familiarity with the financial institution’s business does not itself establish the technical capability to operate agents, and technical capability does not establish knowledge of financial processes. Respondents described both types of knowledge gap. Some considered the vendor knowledgeable about financial services but inexperienced in agent operations. Others described technically capable teams learning the financial business from financial institution documents. These accounts do not establish how often firms possessed both capabilities. Several accounts also described expertise concentrated in individual staff members and limited technical support from senior management. > “our depth in market abuse work sits in about four people company-wide, and I get them for a week at a time when I ask nicely.” -Engagement Manager, ITO, Regulatory, audit, tax and legal ### 3.1 Depth of Vendor Knowledge of the Financial Industry Indicator C120 · Rate the depth of your employer’s knowledge of the financial industry · Full statistical analysis › Some respondents described learning the financial business during delivery, while favorable accounts of vendor industry knowledge located the weakness in agent operations. The mean rating was 3.6 and the median 4.0 (n = 802 respondents). On canceled engagements, the vendor's industry knowledge was rated lower than on active engagements in any stage (mean 2.4, n = 302 respondents). Figure 2.51. Depth of Vendor Knowledge of the Financial Industry: associations with the study measures Scroll horizontally to view the complete figure. Depth of vendor knowledge of the financial industry: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.71 [0.59, 0.79] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 | | Intent to leave | | r = −0.60 [−0.68, −0.52] 755 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66083393290000003 | 1 | | 95% confidence interval | 1 | -0.43899275170000002 | 1 | | Reported association | 0 | -0.55986568439999995 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.58713775769999998 | 1 | | 95% confidence interval | 1 | 0.79475395630000001 | 1 | | Reported association | 0 | 0.70579095719999996 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.79557042290000002 | 1 | | 95% confidence interval | 1 | 0.88743836679999999 | 1 | | Reported association | 0 | 0.84774502620000003 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.67598163150000001 | 1 | | 95% confidence interval | 1 | -0.52242747560000002 | 1 | | Reported association | 0 | -0.60443399499999995 | 1 | Note: Survey question: “Rate the depth of your employer’s knowledge of the financial industry”. Distressed ratings: 392 of 802 answers (48.88%). Group means and numeric-answer bases: Individual contributor (n = 619): 3.52. Engagement manager (n = 150): 3.94. Portfolio director (n = 31): 4.65. Build-Out (n = 166): 4.39. Transition/Cut-Over (n = 235): 4.31. Production (n = 99): 4.59. Canceled (n = 302): 2.41. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. All four core study measures were associated with the depth of the vendor's knowledge of the financial industry, and none of the applicable robustness checks overturned an association. Deeper industry knowledge was associated with a lower cancellation rate (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Customer satisfaction among active engagements was higher where the vendor's industry knowledge was rated deeper (r = 0.71, 95% CI 0.59 to 0.79, n = 94 engagements). The engagement indicator mean, computed without the industry-knowledge indicator, was higher as well (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated the vendor's industry knowledge higher reported lower intent to leave (r = −0.60, cluster bootstrap 95% CI −0.68 to −0.52, n = 755 respondents). Distressed ratings came from 392 of the 802 answers (48.88%), and the engagement mean was distressed on 60 of the 150 engagements. The written responses addressing the vendor's industry knowledge numbered 64, all in comment on the vendor, with 45 negative and 18 positive. The negative responses described delivery team members who did not know the financial business they served and learned it on live work, in some accounts from the financial institution's own documents. The positive responses held that the firm's domain knowledge was sound and placed the gap in running agents. > “In week three I asked for two analysts who had done entitlement reviews before. What arrived were two people who had completed the internal AI course and had never seen a bank access review. I spent a fortnight teaching segregation of duties instead of running the engagement.” -Engagement Manager, ITO, Technology engineering and operations > “We have the audit domain people, that has never been the gap here … Where we are thin is running the same agents past a pilot. This was the first continuous auditing build any of us had taken beyond a demo environment, and it showed in how long it took us to work out what to do when a control test failed for the wrong reason.” -Engagement Manager, KPO, Regulatory, audit, tax and legal > “One thing plainly, the people my employer puts onto a deposits account do not know deposits. For that reason I spend the first month of every intake explaining what a hold is, what a dormancy flag is.” -Individual Contributor, BPO, Payments, cards and disputes > “above him nobody has worked in a bank, because of that the training deck they gave us was written for a call centre, not for payroll cases” -Individual Contributor, BPO, Corporate and back-office functions > “we were given the tool and a two day session, nothing about private capital in it. nobody on our side could explain what a management fee offset is, i had to ask the client team through my lead” -Individual Contributor, BPO, Trade, settlement and reconciliation More negative written sentiment about industry knowledge was associated with higher intent to leave, but no statistically significant association was detected with the engagement-level measures. ### 3.2 Depth of Vendor Knowledge of Agentic AI Indicator C131 · Rate the depth of your employer’s knowledge of agentic AI · Full statistical analysis › Vendor knowledge of agentic AI received the lowest mean rating among the vendor capability indicators. The mean rating was 3.58, the lowest of the 20 indicators of vendor capability, and the median 3.0 (n = 807 respondents). The vendor's agentic knowledge received a mean rating of 2.4 on canceled engagements (n = 308 respondents), lower than on active engagements in any stage. Figure 2.52. Depth of Vendor Knowledge of Agentic AI: associations with the study measures Scroll horizontally to view the complete figure. Depth of vendor knowledge of agentic AI: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.57 [−0.67, −0.45] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.68 [0.56, 0.78] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.59 [−0.66, −0.51] 758 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66592652240000005 | 1 | | 95% confidence interval | 1 | -0.44630581000000003 | 1 | | Reported association | 0 | -0.56607822320000001 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.5551239845 | 1 | | 95% confidence interval | 1 | 0.77657267649999995 | 1 | | Reported association | 0 | 0.68112921299999996 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80960173840000005 | 1 | | 95% confidence interval | 1 | 0.89553105749999995 | 1 | | Reported association | 0 | 0.85847751520000004 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.65550501800000005 | 1 | | 95% confidence interval | 1 | -0.50976727960000001 | 1 | | Reported association | 0 | -0.58517060180000002 | 1 | Note: Survey question: “Rate the depth of your employer’s knowledge of agentic AI”. Distressed ratings: 416 of 807 answers (51.55%). Group means and numeric-answer bases: Individual contributor (n = 625): 3.43. Engagement manager (n = 150): 3.93. Portfolio director (n = 30): 4.70. Build-Out (n = 164): 4.28. Transition/Cut-Over (n = 235): 4.21. Production (n = 100): 4.57. Canceled (n = 308): 2.40. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The depth of the vendor's knowledge of agentic AI was associated with all four core study measures, and each association held under the applicable robustness checks. Engagements whose vendor was rated more knowledgeable about agents had a lower cancellation rate (r = −0.57, 95% CI −0.67 to −0.45, n = 150 engagements). Among active engagements, customer satisfaction was higher where agentic knowledge was rated deeper (r = 0.68, 95% CI 0.56 to 0.78, n = 94 engagements). The engagement indicator mean, computed without the agentic-knowledge indicator, was higher where that knowledge was rated deeper (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Intent to leave was lower where respondents rated the vendor's agentic knowledge higher (r = −0.59, cluster bootstrap 95% CI −0.66 to −0.51, n = 758 respondents). Distressed ratings came from 416 of the 807 answers (51.55%), and the engagement mean was distressed on 63 of the 150 engagements. Of the 26 written responses addressing agentic knowledge, 18 were negative. All appeared in comments about the vendor. Negative accounts described limited technical knowledge above the delivery-team level, brief initial training, and learning during live work. > “The problem is nobody above us really know these agents, so when we get stuck we solve it ourselves from Google. It make my team tired, we are learning on live tickets and there is no one to ask” -Individual Contributor, ITO, Technology engineering and operations > “They put us through the training and it was decent, I'll say that, better than the last place. But nobody above me actually knows this stuff to the depth the work needs. We're figuring out the agent side on the desk.” -Individual Contributor, BPO, Trade, settlement and reconciliation > “Nobody from my company could explain what the agent had done. i asked my lead, he asked somebody else, then it came back to me.” -Individual Contributor, ITO, Financial crime and client onboarding > “They taught me the banking side properly. The agent side we are all guessing” -Individual Contributor, ITO, Technology engineering and operations > “The knowledge sits in perhaps a dozen individuals, which is why I marked the capability question higher than the playbook one; what we lack is not understanding but the written route from a working prototype to a governed sanctions process, and that gap is real.” -Portfolio Director, KPO, Financial crime and client onboarding Written sentiment on agentic knowledge showed no statistically significant association with the study measures. The 26 responses suggest that expertise was concentrated in particular individuals, but the small qualitative sample limits broader inference about management capability. > i left some of these blank, i cannot rate what i do not see. our AI knowledge is good, the people who wrote triage playbook know it well. but scoring training out of ten is not useful, we get one induction then you learn from tickets” INDIVIDUAL CONTRIBUTOR · ITO, Technology engineering and operations ## 4 Live Production Experience and the Operation of Large Numbers of Agents Respondents distinguished limited experience beyond pilots from gaps in monitoring and exception handling as agent populations expanded. Two indicators distinguish experience operating agents beyond pilots from the capability to manage large agent populations, including monitoring and exception handling. The distinction separates prior production exposure from the operational capacity required to sustain delivery at scale. > “Not one person at my firm has done this before. Nobody. They sold agent desk support to a trading bank and then went looking for who knows how to run it, and the answer was nobody, so they put us on it and told us learn. That is not malice. That is a firm out of its depth that will not say so out loud...” -Individual Contributor, ITO, Technology engineering and operations > “But running hundreds of agents at once is different from running ten, and our own alerting for the agents themselves is thin. We watch the client estate closely and the bots not at all.” -Individual Contributor, ITO, Technology engineering and operations ### 4.1 Vendor Experience Running Agents Beyond Pilots Indicator C132 · Rate the depth of your employer’s experience running agentic AI systems in live production, beyond pilots · Full statistical analysis › Limited vendor experience running agents beyond pilots was a recurring concern in accounts of learning during live production work. The mean rating was 3.6 and the median 4.0 (n = 794 respondents). Respondents on canceled engagements rated the vendor's production experience lower than respondents on active engagements in any stage (mean 2.4, n = 300 respondents). Figure 2.53. Vendor Experience Running Agents Beyond Pilots: associations with the study measures Scroll horizontally to view the complete figure. Vendor experience running agents beyond pilots: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.73 [0.62, 0.81] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.87 [0.82, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.60 [−0.67, −0.52] 747 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66243429269999998 | 1 | | 95% confidence interval | 1 | -0.44128773700000001 | 1 | | Reported association | 0 | -0.56181676879999998 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.61572937910000003 | 1 | | 95% confidence interval | 1 | 0.81068272289999999 | 1 | | Reported association | 0 | 0.72757598769999998 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.82213363709999998 | 1 | | 95% confidence interval | 1 | 0.90271121129999998 | 1 | | Reported association | 0 | 0.86802705140000003 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66707638229999999 | 1 | | 95% confidence interval | 1 | -0.52050275909999999 | 1 | | Reported association | 0 | -0.59827123059999998 | 1 | Note: Survey question: “Rate the depth of your employer’s experience running agentic AI systems in live production, beyond pilots”. Distressed ratings: 383 of 794 answers (48.24%). Group means and numeric-answer bases: Individual contributor (n = 611): 3.55. Engagement manager (n = 150): 3.78. Portfolio director (n = 31): 4.65. Build-Out (n = 159): 4.41. Transition/Cut-Over (n = 237): 4.24. Production (n = 98): 4.66. Canceled (n = 300): 2.43. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The depth of the vendor's experience running agents beyond pilots was associated with cancellation, customer satisfaction, the engagement indicator mean, and intent to leave, and every association survived the applicable robustness checks. Engagements whose vendor had deeper production experience had lower observed cancellation rates (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Customer satisfaction among active engagements was higher where production experience was rated deeper (r = 0.73, 95% CI 0.62 to 0.81, n = 94 engagements). So was the engagement indicator mean computed without the production-experience indicator (r = 0.87, 95% CI 0.82 to 0.90, n = 150 engagements). Respondents who rated production experience deeper reported lower intent to leave (r = −0.60, cluster bootstrap 95% CI −0.67 to −0.52, n = 747 respondents). Distressed ratings came from 383 of the 794 answers (48.24%), and the engagement mean was distressed on 62 of the 150 engagements. Of the 54 written responses addressing production experience, 51 were negative, including 42 assigned the most negative sentiment score. Negative comments about the vendor described limited prior production experience, brief preparation, and learning during live work for the financial institution. Some respondents perceived a gap between the experience implied during sales and the capability available during delivery. > “Nobody in my company has done this before. Not in real production. We are told the model will triage the alerts and we only look at what it is not sure about, but on my shift i am running maybe half of them again by hand because i do not trust what it closed... a closed alert on card fraud is not same as closed ticket for a printer.” -Individual Contributor, ITO, Technology engineering and operations > “when we started, nobody in my team had seen an agent run a case queue before, we learnt on the job. What must change practically is two things. First, someone senior with production experience should sit with the team for the first month, not a deck at kickoff.” -Individual Contributor, KPO, Financial crime and client onboarding > “When we won this, the induction was two days of reading and then we were on live composites. Nobody in the firm had run agents in production on a compliance account before -- we found that out afterwards.” -Engagement Manager, KPO, Trade, settlement and reconciliation > “To be fair nobody in this company has run credit early-warning agents before... they are learning same as me. But they sold it like they have. And training is two videos, that part I cannot defend.” -Individual Contributor, KPO, Credit, underwriting and suitability > “What we do not have is people who have run agents in live production rather than in pilots” -Engagement Manager, BPO, Payments, cards and disputes Written sentiment on production experience showed no statistically significant association with the study measures. The predominance of negative responses restricted variation in sentiment and may have reduced the analysis’s ability to detect an association. The accounts nevertheless identify a recurring concern about limited experience before production deployment. ### 4.2 Operational Capacity for Large Agent Populations Indicator C133 · Rate your employer’s ability to operate large numbers of agents in production, including monitoring and exception handling · Full statistical analysis › Accounts of operating large agent populations identified gaps in monitoring ownership, exception handling, and escalation as the number of agents increased. The mean rating was 3.7 and the median 4.0 (n = 798 respondents). The operation of large numbers of agents was rated lowest on canceled engagements, at a mean of 2.4 (n = 297 respondents), lower than on active engagements in any stage. Figure 2.54. Operational Capacity for Large Agent Populations: associations with the study measures Scroll horizontally to view the complete figure. Operational capacity for large agent populations: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.59 [−0.68, −0.47] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.70 [0.58, 0.79] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.87 [0.83, 0.91] 150 engagements p < 0.001 | | Intent to leave | | r = −0.61 [−0.67, −0.53] 750 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68372714489999997 | 1 | | 95% confidence interval | 1 | -0.47210041489999999 | 1 | | Reported association | 0 | -0.5878829887 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.58218636960000003 | 1 | | 95% confidence interval | 1 | 0.79196611049999999 | 1 | | Reported association | 0 | 0.70199537369999998 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.82636221949999999 | 1 | | 95% confidence interval | 1 | 0.90512389329999998 | 1 | | Reported association | 0 | 0.87124166550000004 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.6747845911 | 1 | | 95% confidence interval | 1 | -0.53286313470000002 | 1 | | Reported association | 0 | -0.60808720039999997 | 1 | Note: Survey question: “Rate your employer’s ability to operate large numbers of agents in production, including monitoring and exception handling”. Distressed ratings: 379 of 798 answers (47.49%). Group means and numeric-answer bases: Individual contributor (n = 616): 3.58. Engagement manager (n = 150): 3.87. Portfolio director (n = 30): 4.90. Build-Out (n = 168): 4.55. Transition/Cut-Over (n = 234): 4.35. Production (n = 99): 4.45. Canceled (n = 297): 2.42. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. Every association of the vendor's ability to operate large numbers of agents with the four core study measures held, in each of the applicable robustness checks. Engagements whose vendor operated its agents better had a lower cancellation rate (r = −0.59, 95% CI −0.68 to −0.47, n = 150 engagements). Among active engagements, customer satisfaction was higher where agent operations were rated stronger (r = 0.70, 95% CI 0.58 to 0.79, n = 94 engagements). The engagement indicator mean, computed without the operations indicator, was higher as well (r = 0.87, 95% CI 0.83 to 0.91, n = 150 engagements). Respondents who rated agent operations higher reported lower intent to leave (r = −0.61, cluster bootstrap 95% CI −0.67 to −0.53, n = 750 respondents). Distressed ratings came from 379 of the 798 answers (47.49%), and the engagement mean was distressed on 64 of the 150 engagements. Of the 37 written responses addressing large-scale agent operations, 29 were negative. Respondents described unassigned monitoring responsibilities, failures discovered after the event, and difficulty expanding from a small number of agents to hundreds. Positive accounts described effective monitoring arrangements. > “We had two agents provisioning at once in same tenant and nobody on our side was watching the queue, one of them looped and raised forty duplicate accounts before a colleague noticed. Took us whole shift to unwind. There is no runbook for that, we wrote one ourselves after.” -Individual Contributor, ITO, Technology engineering and operations > “we were given the assist tooling and a two-hour handover call, and no operations model behind it. when an agent misfires at 2am there is nobody on our side rostered to notice, i am the escalation path, for that reason i keep my phone next to me.” -Engagement Manager, CXM, Corporate and back-office functions > “nobody from my company is watching the agent while it runs, one week it keeps closing tickets as resolved which are not resolved, we find out from angry employees and not from any dashboard. the people who trained us do not know banking either.” -Individual Contributor, CXM, Corporate and back-office functions > “We run agents on the release pipeline and there is no proper dashboard from our side, if one starts misbehaving at 2am we dont hear it first, the bank tells us.” -Individual Contributor, ITO, Technology engineering and operations > “Two months back we had an agent loop on a sandbox endpoint over a weekend, and our own operations desk caught it on the monitoring before the client noticed, and had it stopped within the hour.” -Engagement Manager, CXM, Technology engineering and operations More negative written sentiment about agent operations was associated with higher intent to leave. Respondents’ accounts identify monitoring ownership, alerting, and out-of-hours escalation as operational concerns. > Problem is training. two hours video on agents, nothing on credit side, my team learn early warning rules from each other. Running agents in production they do well now, monitoring is better than start. But when an alert goes wrong nobody above wants to own it” INDIVIDUAL CONTRIBUTOR · KPO, Credit, underwriting and suitability ## 5 Talent Supply, Development, and Training Reported talent shortages and brief or insufficiently relevant training left some delivery teams relying on a small number of experienced colleagues. Five indicators assess the supply of qualified staff, process and leadership development, training investment, practical agentic skills, and financial-industry knowledge. Together, they distinguish recruitment capacity from the development of skills within the existing workforce. The written evidence describes brief or generic initial training, learning during live delivery, prolonged vacancies, and reliance on a small number of experienced staff. > “When our automation lead went on leave for two weeks there was nobody to cover, we just paused the closure rules testing till he came back. That is my only real complaint, otherwise the support from my manager is good.” -Individual Contributor, ITO, Technology engineering and operations Part 4 examines workforce experience and intent to leave, including the implications of knowledge held in personal notes or informal peer networks. > “There is no training on any of this, so we keep our own notes file, one page per Guidewire config we automated, what the agent gets wrong and what to check after. Every new joiner gets that file on day one. Because of that nobody asks the training team for anything now.” -Individual Contributor, ITO, Client servicing and communications ### 5.1 Supply of Qualified Agentic Talent Indicator C122 · Rate your employer’s current supply of qualified agentic AI talent against what its engagements need · Full statistical analysis › Respondents described shortages of qualified agentic AI personnel, slow replacement, and expertise concentrated in a small number of people. The mean rating was 3.7 and the median 4.0 (n = 802 respondents). The mean rating of the supply of talent on canceled engagements, 2.5 (n = 304 respondents), was lower than on active engagements in any stage. Figure 2.55. Supply of Qualified Agentic Talent: associations with the study measures Scroll horizontally to view the complete figure. Supply of qualified agentic talent: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.59 [−0.68, −0.47] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.71 [0.59, 0.80] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.87 [0.83, 0.91] 150 engagements p < 0.001 | | Intent to leave | | r = −0.62 [−0.68, −0.54] 754 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68481047279999996 | 1 | | 95% confidence interval | 1 | -0.47368203460000002 | 1 | | Reported association | 0 | -0.58921451589999996 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.59318529519999996 | 1 | | 95% confidence interval | 1 | 0.79814713820000005 | 1 | | Reported association | 0 | 0.71041759270000004 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.82710997819999998 | 1 | | 95% confidence interval | 1 | 0.9055500112 | 1 | | Reported association | 0 | 0.87180971949999997 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68181522650000004 | 1 | | 95% confidence interval | 1 | -0.54038300189999999 | 1 | | Reported association | 0 | -0.61634134549999997 | 1 | Note: Survey question: “Rate your employer’s current supply of qualified agentic AI talent against what its engagements need”. Distressed ratings: 382 of 802 answers (47.63%). Group means and numeric-answer bases: Individual contributor (n = 620): 3.57. Engagement manager (n = 150): 4.01. Portfolio director (n = 31): 4.61. Build-Out (n = 164): 4.53. Transition/Cut-Over (n = 241): 4.31. Production (n = 93): 4.65. Canceled (n = 304): 2.46. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. Each core study measure was associated with the supply of qualified agentic talent, and every association withstood the applicable robustness checks. Cancellation was less common where the supply of talent was rated adequate (r = −0.59, 95% CI −0.68 to −0.47, n = 150 engagements). Active engagements whose vendor had a better supply of talent reported higher customer satisfaction (r = 0.71, 95% CI 0.59 to 0.80, n = 94 engagements). The engagement indicator mean computed without the talent indicator was closely associated with the rating (r = 0.87, 95% CI 0.83 to 0.91, n = 150 engagements). Respondents who rated the supply of talent higher reported lower intent to leave (r = −0.62, cluster bootstrap 95% CI −0.68 to −0.54, n = 754 respondents). Distressed ratings came from 382 of the 802 answers (47.63%), and the engagement mean was distressed on 58 of the 150 engagements. The written responses addressing the supply of talent numbered 158, and 144 were negative. Every respondent who addressed it in naming the hardest challenges named a shortage of agentic talent. The comments on the vendor described teams staffed with people who had never performed agentic tasks, slow replacement, and knowledge held by two or three people shared across accounts. > “We are three testers and only one of us has done agent work before … when I need someone senior to explain why the agent behaves one way on Monday and another way on Thursday, there is nobody in the company to ask, we post in a chat group and wait two days.” -Individual Contributor, ITO, Technology engineering and operations > “When two of our people moved off in March we waited nearly six weeks for replacements, and the ones who came dont know the queue at all, we teach them from scratch ourselves.” -Individual Contributor, CXM, Client servicing and communications > “It felt slow when two people rolled off in month two, replacements took nearly six weeks and neither had done a close before. We managed, but my lead did the training himself in the evenings, thats not a plan.” -Individual Contributor, BPO, Trade, settlement and reconciliation > “In March the failover config drifted on the tick capture box and we lose about two hours before somebody who actually knows that kit comes on the bridge... There are maybe three people here who know this environment properly.” -Individual Contributor, ITO, Technology engineering and operations > “nobody they staffed had done a network certification before, me included. … what the numbers dont show is we were reading the processors spec ourselfs at night to keep up, thats not a training program thats just people covering.” -Individual Contributor, ITO, Payments, cards and disputes > “Our team is full of freshers. We need senior workers with some agentic AI experience. We've had open reqs for months, and can't find anyone. I will have to tell the customer we will miss targets if we can't find them by the end of the month.” -Portfolio Director, BPO, Financial crime and client onboarding More negative engagement-level sentiment about talent supply was associated with higher cancellation rates. More negative respondent-level sentiment was associated with higher intent to leave. Respondents identified three recurring concerns: inexperienced assignments, slow recruitment, and concentrated expertise. ### 5.2 Process and Leadership Development Programs Indicator C118 · Rate the effectiveness of your employer’s process and leadership development programs · Full statistical analysis › Process and leadership development received less favorable ratings on canceled engagements, while the small written-answer set distinguished classroom instruction from preparation for delivery. The mean rating was 3.7 and the median 4.0 (n = 803 respondents). Respondents on canceled engagements rated the programs lower than respondents on active engagements in any stage (mean 2.4, n = 306 respondents). Figure 2.56. Process and Leadership Development Programs: associations with the study measures Scroll horizontally to view the complete figure. Process and leadership development programs: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.59 [−0.69, −0.48] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.69 [0.57, 0.78] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 | | Intent to leave | | r = −0.61 [−0.67, −0.53] 753 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.6878993202 | 1 | | 95% confidence interval | 1 | -0.47819913009999998 | 1 | | Reported association | 0 | -0.59301389630000001 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.56684791170000004 | 1 | | 95% confidence interval | 1 | 0.78327401630000004 | 1 | | Reported association | 0 | 0.69019402019999998 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80362114790000005 | 1 | | 95% confidence interval | 1 | 0.89208863289999996 | 1 | | Reported association | 0 | 0.85390821760000002 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.6724005072 | 1 | | 95% confidence interval | 1 | -0.52908109199999998 | 1 | | Reported association | 0 | -0.60547967439999995 | 1 | Note: Survey question: “Rate the effectiveness of your employer’s process and leadership development programs”. Distressed ratings: 393 of 803 answers (48.94%). Group means and numeric-answer bases: Individual contributor (n = 621): 3.55. Engagement manager (n = 150): 3.95. Portfolio director (n = 31): 4.68. Build-Out (n = 165): 4.48. Transition/Cut-Over (n = 238): 4.34. Production (n = 94): 4.67. Canceled (n = 306): 2.42. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. All four core study measures were associated with the vendor's process and leadership development programs, and none of the applicable robustness checks overturned an association. More effective programs were associated with a lower observed cancellation rate (r = −0.59, 95% CI −0.69 to −0.48, n = 150 engagements). Customer satisfaction among active engagements was higher where the programs were rated more effective (r = 0.69, 95% CI 0.57 to 0.78, n = 94 engagements). The engagement indicator mean, computed without the programs indicator, was higher too (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Intent to leave was lower among respondents who rated the programs higher (r = −0.61, cluster bootstrap 95% CI −0.67 to −0.53, n = 753 respondents). Distressed ratings came from 393 of the 803 answers (48.94%), and the engagement mean was distressed on 64 of the 150 engagements. Seven written responses addressed development programs. Four were negative. Negative accounts described classroom or slide-based instruction with limited application to agentic delivery. Positive accounts considered the programs adequate but identified weaknesses in accountability for agent errors. This small sample supports illustrative observations only. > “The knowledge is there at the top but it does not come down to us. Leadership programme is a set of slides nobody finishes. And I learnt everything from my team lead” -Individual Contributor, KPO, Credit, underwriting and suitability > “Leadership programs exist but they are classroom, not for this kind of delivery.” -Engagement Manager, BPO, Corporate and back-office functions > “We were put on this desk with two days of induction and the leadership course i was sent on was mostly slides. And nobody above me has run agents live before this.” -Individual Contributor, ITO, Technology engineering and operations The structured ratings provide the broader assessment of development programs. They should not be interpreted as measuring agreement with the specific explanations offered in the seven written responses. ### 5.3 Adequacy of Vendor Investment in Training Indicator C123 · Rate the adequacy of your employer’s investment in training its workforce · Full statistical analysis › Respondents described vendor training that was brief or insufficiently relevant to the assignment, with practical learning supplemented by colleagues during delivery. The mean rating was 3.7 and the median 4.0 (n = 789 respondents). On canceled engagements the investment in training had a mean rating of 2.5 (n = 298 respondents), lower than on active engagements in any stage. Figure 2.57. Adequacy of Vendor Investment in Training: associations with the study measures Scroll horizontally to view the complete figure. Adequacy of vendor investment in training: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.57 [−0.67, −0.45] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.71 [0.59, 0.80] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.59 [−0.67, −0.51] 742 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.67063287819999995 | 1 | | 95% confidence interval | 1 | -0.45309041150000001 | 1 | | Reported association | 0 | -0.5718296907 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.59148998789999996 | 1 | | 95% confidence interval | 1 | 0.79719723669999998 | 1 | | Reported association | 0 | 0.70912162840000004 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.81316314329999995 | 1 | | 95% confidence interval | 1 | 0.89757612620000005 | 1 | | Reported association | 0 | 0.86119482260000002 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66679210789999999 | 1 | | 95% confidence interval | 1 | -0.51077083499999998 | 1 | | Reported association | 0 | -0.59421548369999999 | 1 | Note: Survey question: “Rate the adequacy of your employer’s investment in training its workforce”. Distressed ratings: 388 of 789 answers (49.18%). Group means and numeric-answer bases: Individual contributor (n = 608): 3.52. Engagement manager (n = 150): 3.97. Portfolio director (n = 30): 4.77. Build-Out (n = 161): 4.47. Transition/Cut-Over (n = 230): 4.22. Production (n = 100): 4.54. Canceled (n = 298): 2.49. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The adequacy of the vendor's investment in training was associated with all four core study measures, and each association held under the applicable robustness checks. Engagements whose vendor invested more adequately in training had a lower cancellation rate (r = −0.57, 95% CI −0.67 to −0.45, n = 150 engagements). Among active engagements, customer satisfaction was higher where the investment was rated adequate (r = 0.71, 95% CI 0.59 to 0.80, n = 94 engagements). So was the engagement indicator mean computed without the training-investment indicator (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Respondents who rated the investment higher reported lower intent to leave (r = −0.59, cluster bootstrap 95% CI −0.67 to −0.51, n = 742 respondents). Distressed ratings came from 388 of the 789 answers (49.18%), and the engagement mean was distressed on 64 of the 150 engagements. The investment in training was the subject of 244 written responses, the most of any indicator of vendor capability, nearly all in comment on the vendor. They divided into 177 negative and 62 positive. The negative responses described training of a few hours or days, much of it slides or recordings unrelated to the engagement's business, followed by live work learned from colleagues. The positive responses described training that had improved, and some named a gap that remained in the business or in checking agent output. > “We were given access to the payroll agent and a two hour session on a Friday, recorded, and that was the preparation. Nobody from the practice side sat with us for even one Cycle. My four staff learned it by running it against last quarter file and comparing what came out, mostly on their own time, and I let that happen because there was no other way to be ready.” -Engagement Manager, BPO, Corporate and back-office functions > “training was two days of slides then straight to live cases. so i made my own file. i wrote down every requirement code and what the underwriter actually wants for it, and i share it with two other new joiner on my team. it is not official, nobody ask me to make it … we lose everything when someone resigns because it all sits in their head or in a file like mine.” -Individual Contributor, BPO, Credit, underwriting and suitability > “Second week they put me alone on month end attribution run, nobody to ask. Training was two days recorded video and that is all. That is what i marked down, not my manager, he is also learning same as me” -Individual Contributor, KPO, Trade, settlement and reconciliation > “Here I get one day of induction and a link to a folder of PDFs. Nobody in my team has done valuation work before, we are all learning from the client's old models.” -Individual Contributor, KPO, Corporate and back-office functions > “i got a half day slide deck and a link to a course nobody finished.” -Individual Contributor, ITO, Payments, cards and disputes More negative written sentiment about training investment was associated with higher cancellation rates and higher intent to leave. Respondents’ accounts suggest that informal learning supplemented training they considered brief or insufficiently relevant to their assignments. ### 5.4 Practical Agentic Skills Built by Vendor Training Indicator C124 · Rate how well your employer’s training programs build practical agentic AI skills · Full statistical analysis › Written accounts distinguished practical agentic AI instruction from theoretical courses and training on tools other than those used in the engagement. The mean rating was 3.7 and the median 4.0 (n = 804 respondents). On canceled engagements, the practical value of the training was rated lower than on active engagements in any stage (mean 2.5, n = 308 respondents). Figure 2.58. Practical Agentic Skills Built by Vendor Training: associations with the study measures Scroll horizontally to view the complete figure. Practical agentic skills built by vendor training: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.58 [−0.68, −0.46] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.69 [0.57, 0.78] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 | | Intent to leave | | r = −0.60 [−0.67, −0.52] 755 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.6760405545 | 1 | | 95% confidence interval | 1 | -0.46091726259999999 | 1 | | Reported association | 0 | -0.57845022989999995 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.56583034350000005 | 1 | | 95% confidence interval | 1 | 0.78269436569999995 | 1 | | Reported association | 0 | 0.68940877919999999 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.79907895559999997 | 1 | | 95% confidence interval | 1 | 0.88946727000000003 | 1 | | Reported association | 0 | 0.85043269870000004 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66984186400000001 | 1 | | 95% confidence interval | 1 | -0.52469825670000003 | 1 | | Reported association | 0 | -0.60126706510000005 | 1 | Note: Survey question: “Rate how well your employer’s training programs build practical agentic AI skills”. Distressed ratings: 380 of 804 answers (47.26%). Group means and numeric-answer bases: Individual contributor (n = 621): 3.58. Engagement manager (n = 150): 3.93. Portfolio director (n = 31): 4.61. Build-Out (n = 160): 4.39. Transition/Cut-Over (n = 241): 4.39. Production (n = 95): 4.60. Canceled (n = 308): 2.50. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The applicable robustness checks left intact each association between the practical agentic skills built by training and the four core study measures. Training that built practical skills was associated with less cancellation (r = −0.58, 95% CI −0.68 to −0.46, n = 150 engagements). Customer satisfaction among active engagements was higher where training was rated more practical (r = 0.69, 95% CI 0.57 to 0.78, n = 94 engagements). The engagement indicator mean, computed without the skills indicator, was closely associated with the rating (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated the training's practical value higher reported lower intent to leave (r = −0.60, cluster bootstrap 95% CI −0.67 to −0.52, n = 755 respondents). Distressed ratings came from 380 of the 804 answers (47.26%), and the engagement mean was distressed on 59 of the 150 engagements. The written responses addressing practical agentic skills numbered 54, nearly all in comment on the vendor, with 42 negative and 12 positive. The negative responses described training delivered as slides and theory, training on tools other than the agent in use, and no instruction on checking agent output that varies between runs. The positive responses found the agentic courses practical and named gaps elsewhere. > “The trainings we get are mostly slide decks... you cannot learn agent debugging from that.” -Individual Contributor, ITO, Technology engineering and operations > “here i get access on day one and a link to a video from 2023 which is about a different tool completely. moreover nobody could tell me who to ask when the agent behave strange.” -Individual Contributor, CXM, Corporate and back-office functions > “But the courses are mostly slides and theory... when i sit with the live queue the first week, nothing in the course was showing me what to do when the agent stops halfway.” -Individual Contributor, CXM, Client servicing and communications > “The training was two weeks of slides then straight to the floor... nobody taught us how to judge a bot answer when it comes out different each time.” -Individual Contributor, BPO, Financial crime and client onboarding > “What I can score is the training, which was actually decent, the agentic modules were hands on and I use that stuff daily.” -Individual Contributor, CXM, Corporate and back-office functions Written sentiment about practical skills showed no statistically significant association with the study measures. Respondents distinguished applicable, hands-on instruction from theoretical courses or training on tools other than those used in delivery. ### 5.5 Financial Industry Knowledge Built by Vendor Training Indicator C125 · Rate how well your employer’s training programs build financial industry knowledge · Full statistical analysis › Financial-industry training received the highest mean rating among vendor capability indicators, although accounts also described learning from colleagues and financial institution documents. The mean rating was 3.72, the highest of the 20 indicators of vendor capability, and the median 4.0 (n = 788 respondents). Respondents on canceled engagements gave industry training a mean rating of 2.6 (n = 294 respondents), lower than respondents on active engagements in any stage. Figure 2.59. Financial Industry Knowledge Built by Vendor Training: associations with the study measures Scroll horizontally to view the complete figure. Financial industry knowledge built by vendor training: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.69 [0.56, 0.78] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 | | Intent to leave | | r = −0.57 [−0.64, −0.49] 743 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66271447610000001 | 1 | | 95% confidence interval | 1 | -0.4416898299 | 1 | | Reported association | 0 | -0.56215847060000002 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.56022080460000001 | 1 | | 95% confidence interval | 1 | 0.77949216389999998 | 1 | | Reported association | 0 | 0.68507476850000004 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80124025610000005 | 1 | | 95% confidence interval | 1 | 0.89071532899999994 | 1 | | Reported association | 0 | 0.85208700569999996 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.63994794089999996 | 1 | | 95% confidence interval | 1 | -0.49335495169999999 | 1 | | Reported association | 0 | -0.57212890810000006 | 1 | Note: Survey question: “Rate how well your employer’s training programs build financial industry knowledge”. Distressed ratings: 380 of 788 answers (48.22%). Group means and numeric-answer bases: Individual contributor (n = 607): 3.61. Engagement manager (n = 150): 4.05. Portfolio director (n = 30): 4.20. Build-Out (n = 162): 4.41. Transition/Cut-Over (n = 233): 4.27. Production (n = 99): 4.70. Canceled (n = 294): 2.57. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. Where training built industry knowledge, all four core study measures were more favorable, and none of the applicable robustness checks reversed an association. Cancellation was less common where industry training was rated stronger (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Among active engagements, customer satisfaction was higher where industry training was rated stronger (r = 0.69, 95% CI 0.56 to 0.78, n = 94 engagements). Industry training was also associated with a higher engagement indicator mean computed without it (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Intent to leave was lower among respondents who rated industry training higher (r = −0.57, cluster bootstrap 95% CI −0.64 to −0.49, n = 743 respondents). Distressed ratings came from 380 of the 788 answers (48.22%), and the engagement mean was distressed on 58 of the 150 engagements. The written responses addressing industry training numbered 82, all in comment on the vendor, with 63 negative and 19 positive. The negative responses described training that was generic and contained little or nothing on the financial business the engagement served. Staff learned the business from colleagues or from the financial institution's own documents. The positive responses described industry courses that worked and placed the gap on the agent side. > “nobody in training explained insurance record classes to us, and the trainer himself was reading from slides. So i keep a sheet of which document type keeps which period and i share the same with two colleagues … But the industry part we teach ourselves...” -Individual Contributor, BPO, Corporate and back-office functions > “Training is the one good part, the agent classes are practicle and I use them daily. But there is nothing on credit in those trainings, so that knowledge I pick up on the floor.” -Individual Contributor, KPO, Credit, underwriting and suitability > “Our people finish induction knowing the platform but not the card business — dispute lifecycle, chargeback reason codes, what a provisional credit actually means to cardholder. On voice queue that gap shows inside first month, and my leads end up teaching it case by case at the desk.” -Engagement Manager, CXM, Payments, cards and disputes > “For banking knowledge we do not wait for corporate training now. I run one hour session every friday where senior reconciler walks the team through actual GL break from that week. It is not in any playbook but it works better than the modules, which are generic.” -Engagement Manager, BPO, Trade, settlement and reconciliation > “Half of my batch had never seen a partnership return before joining and we learn on live files.” -Individual Contributor, KPO, Regulatory, audit, tax and legal Written sentiment about industry training showed no statistically significant association with the study measures. Respondents described both useful formal courses and reliance on colleagues, team notes, or financial institution documents to acquire business knowledge. > i did the induction in my first week and after that nothing... two days of slides about agents and then straight onto the account. that is the thing no number here can show. the training is not late, it is simply not there. i learn from the senior sitting next to me and when he is on leave i learn from google, to be candid. one year on capital markets testing now and still nobody has explained to me properly what a settlement fail actually means for the cases i am writing, i had to ask a senior from another team and even he was surprised nobody told me. on top of that when i discuss about training with my lead he says budget, next quarter, same answer since March. the people here are not the problem. my colleagues help me and my lead is trying. but they keep putting people like me on agent testing with two days of slides and then ask why the cases take long to write. i dont mind hard work. i mind being sent in without the basics... anyway. i said the same thing in the last survey also and nothing came of it, so i am not sure why i am typing this one.” INDIVIDUAL CONTRIBUTOR · ITO, Technology engineering and operations ## 6 Delivery Playbook, Quality Assurance, and Platforms Locally developed procedures and supplementary tools were reported where vendor-wide delivery frameworks, quality assurance, or platforms did not meet engagement needs. Three indicators assess the vendor’s delivery framework, quality assurance for variable agent output, and technical platforms. These capabilities concern the transition from a successful demonstration to a governed and repeatable production service. > “The people I have been given are capable, and the firm knows the technology in the abstract, what is missing is anything written down for a finance back office at a private capital client. We were handed a platform built for contact center work and told to make expense processing fit it, and the training that came with it was generic.” -Engagement Manager, BPO, Corporate and back-office functions > “No shared agentic playbook, every rollout reinvents the wheel” -Individual Contributor, BPO, Trade, settlement and reconciliation Respondents described locally developed procedures and tools where firm-wide arrangements were considered inadequate. ### 6.1 Maturity of the Delivery Framework Indicator C126 · Rate the maturity of your employer’s playbook for taking agentic use cases from concept to governed production · Full statistical analysis › Some delivery teams reported developing their own route from concept to governed production rather than receiving an established agentic AI delivery framework. The mean rating was 3.7 and the median 4.0 (n = 807 respondents). Respondents on canceled engagements rated the playbook lower than respondents on active engagements in any stage (mean 2.4, n = 304 respondents). Figure 2.60. Maturity of the Delivery Framework: associations with the study measures Scroll horizontally to view the complete figure. Maturity of the delivery framework: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.59 [−0.68, −0.47] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.70 [0.58, 0.79] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.62 [−0.68, −0.54] 756 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68363244499999998 | 1 | | 95% confidence interval | 1 | -0.47196222110000002 | 1 | | Reported association | 0 | -0.58776661699999999 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.57778485719999995 | 1 | | 95% confidence interval | 1 | 0.78948050270000003 | 1 | | Reported association | 0 | 0.69861558420000003 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80971513809999995 | 1 | | 95% confidence interval | 1 | 0.89559623109999997 | 1 | | Reported association | 0 | 0.85856408009999996 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68412587830000005 | 1 | | 95% confidence interval | 1 | -0.54443575070000005 | 1 | | Reported association | 0 | -0.61872074889999995 | 1 | Note: Survey question: “Rate the maturity of your employer’s playbook for taking agentic use cases from concept to governed production”. Distressed ratings: 392 of 807 answers (48.57%). Group means and numeric-answer bases: Individual contributor (n = 626): 3.54. Engagement manager (n = 150): 3.95. Portfolio director (n = 30): 4.37. Build-Out (n = 166): 4.40. Transition/Cut-Over (n = 236): 4.28. Production (n = 101): 4.69. Canceled (n = 304): 2.40. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. The maturity of the vendor's delivery playbook showed an association with each of the four core study measures that survived all applicable robustness checks. Engagements run with a more mature playbook had lower observed cancellation rates (r = −0.59, 95% CI −0.68 to −0.47, n = 150 engagements). Active engagements with a more mature playbook reported higher customer satisfaction (r = 0.70, 95% CI 0.58 to 0.79, n = 94 engagements). Playbook maturity and the engagement indicator mean, computed without the playbook indicator, were closely associated (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Intent to leave was lower where respondents judged the playbook mature (r = −0.62, cluster bootstrap 95% CI −0.68 to −0.54, n = 756 respondents). Distressed ratings came from 392 of the 807 answers (48.57%), and the engagement mean was distressed on 60 of the 150 engagements. The playbook was the subject of 200 written responses, and 186 of the 200 were negative, 145 with the strongest negative score. Every respondent who addressed it in naming the hardest challenges named the absence or immaturity of a delivery playbook. The comments on the vendor described a route from pilot to governed production that each team invented for itself, or a playbook written for staffed work. > “we are selling ahead of our own playbook. The platform is good — I have no argument there, and our people know cards and payments as well as anyone in this market — but the route from a promising cardholder use case to something a bank's risk function will sign off is still being invented on each engagement rather than followed.” -Portfolio Director, CXM, Payments, cards and disputes > “on my last account we have proper checklist for exceptions, signed off, everybody follow the same one. Here every pod invent its own for the nostro breaks … we are short of anyone who has actually run these agents before, so we learn on the client's work” -Individual Contributor, BPO, Trade, settlement and reconciliation > “No agentic delivery playbook existed, so every escalation was improvised” -Individual Contributor, BPO, Financial crime and client onboarding > “No agentic playbook exists so every new ticket type starts from zero” -Individual Contributor, CXM, Technology engineering and operations > “the training budget is there, we are sent on the course, but nobody internally has run agents at this size before, so we build our own runbook as we go and take the platform vendor's support call every Tuesday” -Engagement Manager, ITO, Technology engineering and operations More negative written sentiment about the delivery framework was associated with cancellation and higher intent to leave. The accounts suggest that some teams developed production procedures during delivery rather than receiving an established framework before deployment. ### 6.2 Quality Assurance for Nondeterministic Agent Output Indicator C127 · Rate your employer’s quality assurance capability for agents whose outputs are not identical every run · Full statistical analysis › Respondents described reliance on individual judgment when quality assurance did not provide a consistent method for checking agent outputs that varied between runs. The mean rating was 3.7 and the median 4.0 (n = 794 respondents). Quality assurance was weakest on canceled engagements, where the mean rating was 2.5 (n = 302 respondents), lower than on active engagements in any stage. Figure 2.61. Quality Assurance for Nondeterministic Agent Output: associations with the study measures Scroll horizontally to view the complete figure. Quality assurance for nondeterministic agent output: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.59 [−0.68, −0.47] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.70 [0.58, 0.79] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.88 [0.83, 0.91] 150 engagements p < 0.001 | | Intent to leave | | r = −0.61 [−0.68, −0.53] 745 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68204411919999997 | 1 | | 95% confidence interval | 1 | -0.46964595749999999 | 1 | | Reported association | 0 | -0.58581539780000003 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.57980906860000003 | 1 | | 95% confidence interval | 1 | 0.79062447179999995 | 1 | | Reported association | 0 | 0.70017058539999999 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.83194601560000003 | 1 | | 95% confidence interval | 1 | 0.9083020597 | 1 | | Reported association | 0 | 0.87548063799999998 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.67845880700000005 | 1 | | 95% confidence interval | 1 | -0.53444158109999995 | 1 | | Reported association | 0 | -0.61092188459999996 | 1 | Note: Survey question: “Rate your employer’s quality assurance capability for agents whose outputs are not identical every run”. Distressed ratings: 377 of 794 answers (47.48%). Group means and numeric-answer bases: Individual contributor (n = 612): 3.57. Engagement manager (n = 150): 3.94. Portfolio director (n = 30): 4.50. Build-Out (n = 159): 4.42. Transition/Cut-Over (n = 235): 4.36. Production (n = 98): 4.66. Canceled (n = 302): 2.45. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. Quality assurance for agent output that varies between runs was associated with cancellation, customer satisfaction, the engagement indicator mean, and intent to leave, and every association held through the applicable robustness checks. Engagements whose vendor had stronger quality assurance had a lower observed cancellation rate (r = −0.59, 95% CI −0.68 to −0.47, n = 150 engagements). Customer satisfaction among active engagements was higher where quality assurance was rated stronger (r = 0.70, 95% CI 0.58 to 0.79, n = 94 engagements). The association with the engagement indicator mean, computed without the quality-assurance indicator, reached the upper end of the category's range (r = 0.88, 95% CI 0.83 to 0.91, n = 150 engagements). Respondents who rated quality assurance higher reported lower intent to leave (r = −0.61, cluster bootstrap 95% CI −0.68 to −0.53, n = 745 respondents). Distressed ratings came from 377 of the 794 answers (47.48%), and the engagement mean was distressed on 61 of the 150 engagements. Quality assurance was the subject of 151 written responses across 12 survey questions, and 126 were negative. Every respondent who addressed it in naming the hardest challenges named quality assurance for agent output. The responses to the operational questions described output that differed between runs on the same input, staff who would not sign it off as it stood, and checking performed by whoever handled the case. > “i've run the same case five times in one afternoon and gotten five different numbers back, so nothing here holds still long enough to actually sign off on, and that's the part that gets to me, not that it's wrong sometimes, but that it's never the same wrong twice, i can't even build a pattern around it.” -Individual Contributor, KPO, Corporate and back-office functions > “Same tools as when i joined, no proper course, no QA method for what the agent produce. When output come different two times for same policy, nobody has told us what to do, we decide ourselves and move on.” -Individual Contributor, BPO, Client servicing and communications > “My employer put me on this Engagement after two weeks classroom only. There is no proper QA guide for when the screening agent gives different result, so my team make our own checklist in a shared sheet. It works but nobody from above ask us for it. I do not mind, I learn a lot.” -Individual Contributor, BPO, Financial crime and client onboarding > “where i mark 5 is checking of output, because same request can give slighty different answer and there is no proper method for that yet. but in truth nobody in market has that method, so i do not blame my company alone for it” -Individual Contributor, CXM, Corporate and back-office functions > “No real QA existed for an agent that gave different answers each run” -Individual Contributor, KPO, Regulatory, audit, tax and legal More negative written sentiment about quality assurance was associated with higher intent to leave. Accounts of rerunning, selecting, or rewriting outputs suggest that some teams relied on individual judgment where a consistent approval method was unavailable. ### 6.3 Optimization of Vendor Platforms for Agentic Outsourcing Indicator C129 · Rate how well the technical platforms your employer builds or licenses are optimized for agentic AI outsourcing · Full statistical analysis › Vendor platforms were reported to require supplementary spreadsheets, mailboxes, or scripts when systems built for other work did not fit agentic delivery. The mean rating was 3.7 and the median 4.0 (n = 794 respondents). The platforms received a mean rating of 2.5 on canceled engagements (n = 298 respondents), lower than on active engagements in any stage. Figure 2.62. Optimization of Vendor Platforms for Agentic Outsourcing: associations with the study measures Scroll horizontally to view the complete figure. Optimization of vendor platforms for agentic outsourcing: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.57 [−0.67, −0.45] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.70 [0.57, 0.79] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.86 [0.81, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.59 [−0.66, −0.51] 743 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66687067990000004 | 1 | | 95% confidence interval | 1 | -0.44766486979999998 | 1 | | Reported association | 0 | -0.56723126509999999 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.57389909930000005 | 1 | | 95% confidence interval | 1 | 0.78728036400000001 | 1 | | Reported association | 0 | 0.69562733889999995 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.80935353889999995 | 1 | | 95% confidence interval | 1 | 0.89538839830000005 | 1 | | Reported association | 0 | 0.85828803970000001 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.65921345980000001 | 1 | | 95% confidence interval | 1 | -0.51100421220000003 | 1 | | Reported association | 0 | -0.58895566710000002 | 1 | Note: Survey question: “Rate how well the technical platforms your employer builds or licenses are optimized for agentic AI outsourcing”. Distressed ratings: 384 of 794 answers (48.36%). Group means and numeric-answer bases: Individual contributor (n = 612): 3.56. Engagement manager (n = 150): 4.03. Portfolio director (n = 30): 4.70. Build-Out (n = 168): 4.41. Transition/Cut-Over (n = 234): 4.24. Production (n = 94): 4.78. Canceled (n = 298): 2.52. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. How well the vendor's platforms were built for agentic outsourcing was associated with all four core study measures, and each association held under the applicable robustness checks. Cancellation was less common on engagements whose vendor platforms were rated better suited (r = −0.57, 95% CI −0.67 to −0.45, n = 150 engagements). Among active engagements, customer satisfaction was higher where the platforms were rated better suited (r = 0.70, 95% CI 0.57 to 0.79, n = 94 engagements). A higher engagement indicator mean, computed without the platform indicator, accompanied better-suited platforms (r = 0.86, 95% CI 0.81 to 0.90, n = 150 engagements). Intent to leave was lower among respondents who rated the platforms higher (r = −0.59, cluster bootstrap 95% CI −0.66 to −0.51, n = 743 respondents). Distressed ratings came from 384 of the 794 answers (48.36%), and the engagement mean was distressed on 63 of the 150 engagements. The written responses addressing the platforms numbered 61, nearly all in comment on the vendor, with 38 negative and 21 positive. The negative responses described platforms built for other work, such as ticket handling or contact centers, and teams that moved the engagement's tasks into spreadsheets, mailboxes, and personal scripts. The positive responses found the platform sound or improved and placed the gap elsewhere. > “Our platform group still hands us tooling built for a staffed service desk, so every report counts tickets a person touched and there is no clean way to show a ticket the bot closed by itself.” -Engagement Manager, ITO, Technology engineering and operations > “The company can say it has platform, but what we use is a shared mailbox and one spreadsheet. Thus every category change is manual for us. i have raised this. My TL raised it also. Also the answer is always that a proper tool is coming next quarter, and next quarter comes and it is same.” -Individual Contributor, CXM, Technology engineering and operations > “The sales team pitched our dev kit we built as being optimized for AI. No one in sales checked if it was. We panicked for a couple of weeks as we validated AI compatibility after contracts were signed.” -Portfolio Director, ITO, Technology engineering and operations > “The tooling we were given for ticket triage was built for a generic helpdesk, not for purchase orders, and i flagged that twice with no result.” -Engagement Manager, CXM, Corporate and back-office functions > “At March close we had to run reconciliation on client tooling because our platform cant hold volume of merchant transactions, and nobody flagged that before we committed to it.” -Engagement Manager, BPO, Trade, settlement and reconciliation Written sentiment about platforms showed no statistically significant association with the study measures. Respondents distinguished platforms suited to their assignments from systems that required supplementary spreadsheets, mailboxes, or scripts. > Two things have got better in the last year. The people coming onto this account now arrive knowing what an agent is and how to write a decent evaluation set - eighteen months ago I was teaching that myself, so the training investment is real and I have rated it accordingly. Tooling to run the agents is also there. The layer above delivery is where nothing has shifted. We still sell this account on a rate card built for bodies. When the pipeline agent absorbs work that used to take two engineers, the commercial conversation that follows is awkward and nobody at my level is given a line to take. I raise it every quarter review and get told it is being worked on. Also, we dont have anyone who really knows how a markets business runs its release calendar - we learn that from the client, which is back to front” ENGAGEMENT MANAGER · ITO, Technology engineering and operations ## 7 Accountability for Agent Error Accounts of costly agent errors differed in whether vendor managers accepted responsibility or assigned it to the analyst on duty. One indicator assesses respondents’ perceptions of the vendor’s willingness to accept accountability for costly agent errors. The analysis distinguishes the structured rating from accounts of how responsibility was allocated after an incident. ### 7.1 Allocation of Accountability for Agent Errors Indicator C130 · Rate your employer’s readiness to take accountability when an agent it operates makes a costly error · Full statistical analysis › Accounts of accountability for costly agent errors contrasted responsibility assigned to the analyst on duty with acceptance of responsibility by vendor managers. The mean rating was 3.7 and the median 4.0 (n = 794 respondents). Respondents on canceled engagements gave the vendor's readiness a mean rating of 2.4 (n = 307 respondents), lower than respondents on active engagements in any stage. Figure 2.63. Allocation of Accountability for Agent Errors: associations with the study measures Scroll horizontally to view the complete figure. Allocation of accountability for agent errors: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.59 [−0.68, −0.47] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.67 [0.54, 0.77] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.87 [0.82, 0.90] 150 engagements p < 0.001 | | Intent to leave | | r = −0.62 [−0.69, −0.54] 744 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68186517229999999 | 1 | | 95% confidence interval | 1 | -0.46938518150000003 | 1 | | Reported association | 0 | -0.58559563579999996 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.54275896990000005 | 1 | | 95% confidence interval | 1 | 0.76945018099999996 | 1 | | Reported association | 0 | 0.67152668439999996 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.82339528640000004 | 1 | | 95% confidence interval | 1 | 0.90343159529999995 | 1 | | Reported association | 0 | 0.86898657369999999 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.68707866630000003 | 1 | | 95% confidence interval | 1 | -0.54211323830000002 | 1 | | Reported association | 0 | -0.61951989269999996 | 1 | Note: Survey question: “Rate your employer’s readiness to take accountability when an agent it operates makes a costly error”. Distressed ratings: 373 of 794 answers (46.98%). Group means and numeric-answer bases: Individual contributor (n = 611): 3.52. Engagement manager (n = 150): 4.03. Portfolio director (n = 31): 4.71. Build-Out (n = 160): 4.56. Transition/Cut-Over (n = 232): 4.32. Production (n = 95): 4.58. Canceled (n = 307): 2.43. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The engagement indicator mean excludes the tested indicator. All four core study measures were associated with the vendor's readiness to take accountability for agent error, and none of the applicable robustness checks overturned an association. Engagements whose vendor was rated more willing to accept accountability for agent errors had lower observed cancellation rates (r = −0.59, 95% CI −0.68 to −0.47, n = 150 engagements). Customer satisfaction among active engagements was higher where the vendor was rated readier to take accountability (r = 0.67, 95% CI 0.54 to 0.77, n = 94 engagements). Readiness to take accountability also accompanied a higher engagement indicator mean, computed without the accountability indicator (r = 0.87, 95% CI 0.82 to 0.90, n = 150 engagements). Respondents who rated the vendor readier to take accountability reported lower intent to leave (r = −0.62, cluster bootstrap 95% CI −0.69 to −0.54, n = 744 respondents). Distressed ratings came from 373 of the 794 answers (46.98%), and the engagement mean was distressed on 63 of the 150 engagements. Of the 38 written responses addressing accountability, 29 were negative and 8 positive. Nearly all appeared in comments about the vendor. Negative accounts described responsibility assigned to the analyst on duty without acceptance by senior management. Positive accounts described team leaders or vendor managers accepting responsibility for agent errors. > “When the assistant gave a wrong leave-accrual answer to about forty new hires, nobody internally could tell me who signs off the correction or who carries it if the client asks for a credit. We escalated and it went sideways for a week.” -Engagement Manager, CXM, Corporate and back-office functions > “They do own their mistakes though. When the Classifier mistagged a whole batch of custody statements nobody tried to pin that on my team.” -Individual Contributor, BPO, Corporate and back-office functions > “top leadership question is pointless for me, i have never seen these people. ask instead who pays when agent breaks a release, that box is not here. my answer would be nobody, it lands on me at 2am” -Individual Contributor, ITO, Payments, cards and disputes > “nobody tells us who answers if an agent mis-rates a file and it goes out wrong, so my team keeps a private checklist before releasing. That worry sits on the analyst, not on the people above us” -Individual Contributor, BPO, Financial crime and client onboarding > “In may the bot closed forty tickets as resolved which was not resolved. And i raise it same day. Two weeks nobody on our side owns it, then we just reopen them ourselves. After that nobody discuss about it again.” -Individual Contributor, ITO, Technology engineering and operations More negative written sentiment about accountability was associated with higher intent to leave. The accounts illustrate differing allocations of responsibility. The structured rating measures perceived willingness to accept accountability rather than the prevalence of either specific practice. ## 8 Vendor Preparedness Relative to Five Years Earlier Respondents' retrospective assessments of vendor preparedness were less favorable on canceled engagements, without establishing measured historical change. A further question asked respondents to compare the vendor’s current preparedness with its preparedness five years earlier. This retrospective assessment measures perceived change, not the transfer of capabilities observed over time. ### 8.1 Retrospective Assessment of Vendor Preparedness Indicator C093 · Compared to 5 years ago, is my employer more or less prepared to manage outsourcing engagements? · Full statistical analysis › Retrospective assessments of vendor preparedness relative to five years earlier were less favorable on canceled engagements than on active engagements in any measured stage. These are retrospective judgments, not longitudinal measurements of change. Its mean rating was 3.7 and its median 4.0 (n = 786 respondents). On canceled engagements, the vendor's preparedness was rated lower than on active engagements in any stage (mean 2.4, n = 293 respondents). Figure 2.64. Retrospective Assessment of Vendor Preparedness: associations with the study measures Scroll horizontally to view the complete figure. Retrospective assessment of vendor preparedness: higher ratings accompanied more favorable outcomes Scroll horizontally to view the complete comparison. | Comparison | Correlation r | Estimate and analysis base | | --- | --- | --- | | Cancellation | | r = −0.56 [−0.66, −0.44] 150 engagements p < 0.001 | | Customer satisfaction | | r = 0.66 [0.52, 0.76] 94 active engagements p < 0.001 | | Indicator mean | | r = 0.85 [0.80, 0.89] 150 engagements p < 0.001 | | Intent to leave | | r = −0.60 [−0.67, −0.52] 737 respondents p < 0.001 | | | −1 0 +1 | | Exact comparison data Cancellation Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.66201781969999995 | 1 | | 95% confidence interval | 1 | -0.44069021809999998 | 1 | | Reported association | 0 | -0.56130891650000003 | 1 | Customer satisfaction Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.52362754570000003 | 1 | | 95% confidence interval | 1 | 0.75831786649999999 | 1 | | Reported association | 0 | 0.65658371959999995 | 1 | Indicator mean Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | 0.79914487860000005 | 1 | | 95% confidence interval | 1 | 0.88950535760000005 | 1 | | Reported association | 0 | 0.85048317259999995 | 1 | Intent to leave Exact chart data | Series | Point index | X | Y | | --- | --- | --- | --- | | 95% confidence interval | 0 | -0.6722355377 | 1 | | 95% confidence interval | 1 | -0.52450165400000004 | 1 | | Reported association | 0 | -0.603250022 | 1 | Note: Survey question: “Compared to 5 years ago, is my employer more or less prepared to manage outsourcing engagements?”. Distressed ratings: 381 of 786 answers (48.47%). Group means and numeric-answer bases: Individual contributor (n = 603): 3.56. Engagement manager (n = 150): 3.97. Portfolio director (n = 31): 4.87. Build-Out (n = 162): 4.63. Transition/Cut-Over (n = 235): 4.27. Production (n = 96): 4.49. Canceled (n = 293): 2.44. r is Pearson’s correlation coefficient, measuring signed linear association from −1 to +1. Whiskers show 95% confidence intervals where reported. Intent-to-leave intervals, where shown, use a cluster bootstrap. The bootstrap resamples engagements to preserve the grouping of respondent answers. Associations are observational. The applicable robustness checks left intact each association between the vendor's preparedness against five years earlier and the four core study measures. Greater preparedness was associated with a lower observed cancellation rate (r = −0.56, 95% CI −0.66 to −0.44, n = 150 engagements). Customer satisfaction among active engagements was higher where preparedness was rated greater (r = 0.66, 95% CI 0.52 to 0.76, n = 94 engagements). The engagement indicator mean computed without the preparedness indicator moved in the same direction (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). Respondents who rated the vendor better prepared reported lower intent to leave (r = −0.60, cluster bootstrap 95% CI −0.67 to −0.52, n = 737 respondents). Distressed ratings came from 381 of the 786 answers (48.47%), and the engagement mean was distressed on 62 of the 150 engagements. Four written responses addressed the comparison, two positive and two negative. They identified changes in areas including hiring, solutions-team support, test environments, outcome-based pricing, and second-review procedures. These accounts are illustrative and cannot establish the prevalence or overall direction of organizational change. > “… not much has improved... the one thing i will say is they finally gave us proper access to sandbox this quarter, before that we were testing on live extracts which was mad. Everything else is same. No training, no playbook, we make it up per engagement and call it method” -Individual Contributor, KPO, Corporate and back-office functions > “Compared with the reconciliations programme I carried two years ago, the firm has become more willing to price an outcome rather than a headcount, and our review capability for work that does not come out identical each run is stronger, there is a functioning second-read process now which we did not have then.” -Portfolio Director, KPO, Regulatory, audit, tax and legal > “We ran a similar recruiting program two years ago with almost no support behind it. This time we had a solutions team on the calls from week one, which is a real change. Staffing the roles still takes longer than it should.” -Engagement Manager, CXM, Corporate and back-office functions ← Co-ManagementAI Technology →