Study Data · Part 4 · Indicator Deep-Dive Analysis

Where did you gain your AI knowledge?

Indicator C017 · By Phil Hatch · Akholi · October 9, 2026

Indicator Specification

Table 1. Indicator specificationTable
FieldValue
Indicator codeC017
Question as askedWhere did you gain your AI knowledge?
Failure categoryNot one of the 100 rated indicators
Place in the surveyRespondent Demographics > 1.2 Experience and Expertise > Credentials
Indicator classRespondent level
Indicator typeCategorical
Response scaleLevels: Vendor and partner training, On the job, Online courses, University coursework, Self-taught, Employer training program
Answering base800 of 850 respondents gave one of the listed answers, across 150 of 150 engagements. None answered I do not know or I prefer to not answer
Non-answersI do not know 0, I prefer to not answer 0, blank 50
Written responses addressing the indicator37 responses from 37 respondents on 33 engagements (0 primary, 37 secondary)

Summary of Findings

Table 2. Key results: association of the indicator with the four study measuresTable
MeasureEffect sizenValuep
Cancellation rateOmega squared, 0 or 1 flag94 engagements−0.0300.792
Customer satisfactionOmega squared22 engagements−0.0270.521
Engagement indicator meanOmega squared94 engagements−0.0400.940
Intent to leaveOmega squared94 engagements−0.0330.871

Of the 850 respondents, 800 answered the question. The answers spread almost evenly across the six sources, from 125 to 140 respondents each: vendor and partner training, on the job, online courses, university coursework, self-taught and the employer's training program. The source did not differ by role, vendor type or any segment family recorded at engagement level.

The source assigned to each engagement was not associated with cancellation (chi-square 2.39, p = 0.792, n = 94 engagements), customer satisfaction among active engagements (Welch F 0.43, p = 0.521, n = 22 engagements), the engagement indicator mean (Welch F 0.24, p = 0.940, n = 94 engagements) or intent to leave (Welch F 0.36, p = 0.871, n = 94 engagements). The engagement indicator mean and intent to leave were equivalent to zero, and every robustness check held.

The written responses addressing the indicator numbered 37, all from individual contributors, on 33 engagements. Every response was secondary. Of the 37 responses, 19 (51.35%) were negative and 14 (37.84%) positive. Whatever source they named in the answer, the respondents who wrote described learning the agents from colleagues, from outside material and on live work, after training of slides and videos.

Distribution of Responses

Table 3. Distribution of responses, respondent levelTable
AnswerRespondentsShare of answered (base 800)Cumulative shareEngagements with at least oneShare of 150 engagementsDistressed
Vendor and partner training14017.50%17.50%9764.67%Not applicable
On the job13617.00%34.50%9261.33%Not applicable
Online courses13617.00%51.50%10368.67%Not applicable
University coursework13216.50%68.00%8959.33%Not applicable
Self-taught13116.38%84.38%8556.67%Not applicable
Employer training program12515.63%100.00%8858.67%Not applicable
Answered, total800100.00%150100.00%

Each engagement was assigned the source given by the largest number of its respondents. Fifty-six engagements were tied and carry no assigned source, and 94 carry one. Every source was the assigned answer on 10 or more engagements.

Coverage and Origin of Written Responses

All 37 responses were secondary. They came from 2 questions outside the indicator's own survey section (C135 with 28 responses and C030 with 9 responses). Each of the nine responses to C030, on what the respondent loves most about the job, listed agentic skill picked up on the job or firsthand.

The 28 responses to C135, the comment field on the employer, described how the respondents learned the agents. Eleven described the training as slides, recorded videos or a recorded session and a PDF, in several cases followed straight by live work, and one said the course came three months after the respondent had started the work. Eleven said the respondent learned from a lead, a colleague or a senior, and two described a note file the team kept and passed to new joiners in place of the official material. Seven described learning alone, from outside videos at night, search, vendor documents or old test cases and broken runs. Eleven said no one senior, or no one above the respondent, knew the agents or had done the work before, so that questions went unanswered for days or came back to the respondent.

Figure 1. Written responses addressing the indicator, by question answered and written score (n = 37 responses)Exhibit
07.8415.6823.5231.36C135 (secondary)1845C030 (secondary)9−10 Strongly negative−5 Negative0 Neutral+5 Positive+10 Strongly positive
Note: Each bar is one survey question. Every question lies outside the indicator's survey section, so every response is secondary. Segments wide enough to hold a label carry their count. C135 asked: “Add any comments on your employer that the ratings in this section do not capture.” C030 asked: “What are the top 3 things that you love most about your job?”
Table 4. Written responses by originTable
OriginResponsesRespondentsEngagementsNegativeStrongest negative (−10)PositiveMean written score
Primary000000None
Secondary37373319 of 37 (51.35%)1 of 37 (2.70%)14 of 37 (37.84%)−0.8
All responses37373319 of 37 (51.35%)1 of 37 (2.70%)14 of 37 (37.84%)−0.8

Distribution of Written Sentiment

Figure 2. Distribution of written sentiment (n = 37 responses)Exhibit
0%25%50%75%100%−10 Strongly negative1 (2.70%)−5 Negative18 (48.65%)0 Neutral4 (10.81%)+5 Positive14 (37.84%)+10 Strongly positive0 (0.00%)Share of responses
Note: Column height is the share of the 37 responses. Labels give the count and the share.
Table 5. Summary statistics of written sentimentTable
StatisticValueBasis
Mean written score−0.895% CI −2.4 to 0.8, cluster bootstrap by engagement, 10,000 draws, n = 37 responses
Median written score−5.0n = 37 responses
Negative responses19 of 37 (51.35%)95% CI 34.29% to 68.29% of 37 responses, cluster bootstrap by engagement
Strongest negative responses (−10)1 of 37 (2.70%)Statements of totality, constancy or serious harm
Positive responses14 of 37 (37.84%)+5 or +10
Net balance−13.51 percentage pointsShare positive minus share negative, base 37 responses
Respondent-level mean−0.8Mean of each respondent's own mean written score, n = 37 respondents

Of the 37 responses, 19 (51.35%, 95% CI 34.29% to 68.29%) were negative and 1 carried the strongest negative score of −10. Positive responses numbered 14 (37.84%), and 4 were neutral. The responses had a mean written score of −0.8 (95% CI −2.4 to 0.8, n = 37 responses on 33 engagements) and a median of −5.0.

Every response to C030 was positive. The responses to C135 were mostly negative (19 of 28), Of their five positive responses, four paired thin training with a lead, a colleague or the team's own notes that made up for it, and the other found the agentic courses practical and placed the gap in banking knowledge.

Descriptive Statistics by Segment

Table 6. Descriptive statistics by segmentTable
SegmentBaseAnsweredAnswer rateEngagementsVendor and partner training, shareOn the job, shareOnline courses, share
Study set85080094.12% of 85015017.50%17.00%17.00%
Outsourcing model
CXM23222195.26% of 2324217.19%17.65%13.57%
ITO24423094.26% of 2444015.65%18.26%16.09%
BPO20919593.30% of 2093715.90%17.44%21.54%
KPO16515493.33% of 1653122.73%13.64%17.53%
Institution type
Asset management13212292.42% of 1322417.21%15.57%17.21%
Wealth management and private banking14013596.43% of 1402311.85%15.56%20.00%
Investment banking (advisory and capital-markets origination)1059691.43% of 1051916.67%17.71%18.75%
Insurance (life, property and casualty, health)787494.87% of 781420.27%18.92%10.81%
Commercial and business banking827692.68% of 821319.74%15.79%13.16%
Payments and merchant acquiring (networks, acquiring, processing)716895.77% of 711316.18%14.71%17.65%
Credit card and payments issuing736994.52% of 731223.19%23.19%17.39%
Retail and consumer banking736893.15% of 731222.06%11.76%16.18%
Markets and trading (sales and trading, global markets)575698.25% of 571117.86%14.29%19.64%
Private equity and private capital393692.31% of 39913.89%30.56%16.67%
Process family
Corporate and back-office functions24823092.74% of 2484721.74%15.22%15.22%
Technology engineering and operations20419294.12% of 2043514.06%20.83%17.19%
Client servicing and communications949297.87% of 941614.13%16.30%13.04%
Trade, settlement and reconciliation807796.25% of 801514.29%22.08%22.08%
Payments, cards and disputes737095.89% of 731218.57%20.00%15.71%
Credit, underwriting and suitability575494.74% of 57920.37%12.96%11.11%
Regulatory, audit, tax and legal514588.24% of 51911.11%11.11%24.44%
Financial crime and client onboarding434093.02% of 43725.00%7.50%27.50%
Class of work
LOB Operational45142995.12% of 4517617.25%16.55%17.02%
Corporate Functional39937192.98% of 3997417.79%17.52%16.98%
Engagement status and lifecycle stage
Active, Build-Out17516292.57% of 1753119.14%16.67%17.28%
Active, Transition/Cut-Over25023594.00% of 2504317.87%14.89%17.02%
Active, Production1039592.23% of 1032014.74%18.95%20.00%
Canceled32230895.65% of 3225617.21%18.18%15.91%
Respondent role
Individual Contributor66761992.80% of 66715017.77%17.93%15.83%
Engagement Manager150150100.00% of 15015016.67%12.67%24.00%
Portfolio Director312993.55% of 312917.24%20.69%6.90%
Customer Management Executive222 of 22000
Vendor type
IT services major31629292.41% of 3165915.41%15.07%15.41%
BPO specialist19818894.95% of 1983718.09%15.96%21.28%
Consulting firm11711396.58% of 1172321.24%19.47%16.81%
Boutique or specialist provider877990.80% of 871711.39%24.05%16.46%
Pure-play agentic AI firm757296.00% of 751420.83%19.44%16.67%
Not reported, recovered by engagement575698.25% of 574723.21%12.50%12.50%
Table 7. Descriptive statistics by segment, remaining answer levelsTable
SegmentBaseAnsweredAnswer rateEngagementsUniversity coursework, shareSelf-taught, shareEmployer training program, share
Study set85080094.12% of 85015016.50%16.38%15.63%
Outsourcing model
CXM23222195.26% of 2324216.74%20.36%14.48%
ITO24423094.26% of 2444018.70%15.22%16.09%
BPO20919593.30% of 2093717.95%12.82%14.36%
KPO16515493.33% of 1653111.04%16.88%18.18%
Institution type
Asset management13212292.42% of 1322419.67%16.39%13.93%
Wealth management and private banking14013596.43% of 1402317.04%17.04%18.52%
Investment banking (advisory and capital-markets origination)1059691.43% of 1051914.58%18.75%13.54%
Insurance (life, property and casualty, health)787494.87% of 781425.68%9.46%14.86%
Commercial and business banking827692.68% of 821315.79%25.00%10.53%
Payments and merchant acquiring (networks, acquiring, processing)716895.77% of 71137.35%22.06%22.06%
Credit card and payments issuing736994.52% of 731211.59%8.70%15.94%
Retail and consumer banking736893.15% of 731219.12%16.18%14.71%
Markets and trading (sales and trading, global markets)575698.25% of 571114.29%16.07%17.86%
Private equity and private capital393692.31% of 39916.67%8.33%13.89%
Process family
Corporate and back-office functions24823092.74% of 2484712.17%19.57%16.09%
Technology engineering and operations20419294.12% of 2043515.10%15.10%17.71%
Client servicing and communications949297.87% of 941622.83%19.57%14.13%
Trade, settlement and reconciliation807796.25% of 801519.48%9.09%12.99%
Payments, cards and disputes737095.89% of 731218.57%12.86%14.29%
Credit, underwriting and suitability575494.74% of 57920.37%14.81%20.37%
Regulatory, audit, tax and legal514588.24% of 51920.00%24.44%8.89%
Financial crime and client onboarding434093.02% of 43715.00%10.00%15.00%
Class of work
LOB Operational45142995.12% of 4517618.41%16.08%14.69%
Corporate Functional39937192.98% of 3997414.29%16.71%16.71%
Engagement status and lifecycle stage
Active, Build-Out17516292.57% of 1753115.43%11.73%19.75%
Active, Transition/Cut-Over25023594.00% of 2504317.87%18.30%14.04%
Active, Production1039592.23% of 1032017.89%13.68%14.74%
Canceled32230895.65% of 3225615.58%18.18%14.94%
Respondent role
Individual Contributor66761992.80% of 66715016.64%15.83%15.99%
Engagement Manager150150100.00% of 15015016.00%18.00%12.67%
Portfolio Director312993.55% of 312913.79%20.69%20.69%
Customer Management Executive222 of 22101
Vendor type
IT services major31629292.41% of 3165918.15%18.15%17.81%
BPO specialist19818894.95% of 1983716.49%16.49%11.70%
Consulting firm11711396.58% of 1172311.50%8.85%22.12%
Boutique or specialist provider877990.80% of 871716.46%20.25%11.39%
Pure-play agentic AI firm757296.00% of 751416.67%13.89%12.50%
Not reported, recovered by engagement575698.25% of 574717.86%19.64%14.29%

Each share is computed on the respondents who answered in that row. Rows with fewer than 10 respondents report counts only.

Tests of Difference Between Segments

Table 8. Tests of difference between segmentsTable
Segment familyUnitChi-squaredfpHolm pCramer's VTest for pLevels excluded
Outsourcing modelEngagement17.66150.2881.0000.25Monte Carlo, simulatedNone
Institution typeEngagement4.88100.9241.0000.24Monte Carlo, simulateda
Process familyEngagement2.3150.8101.0000.21Monte Carlo, simulatedb
Class of workEngagement2.7850.7341.0000.17chi-squareNone
Engagement status and lifecycle stageEngagement13.57150.5791.0000.22Monte Carlo, simulatedNone
Respondent roleRespondent11.04100.3541.0000.08Monte Carlo, simulatedCustomer Management Executive, n = 2
Vendor typeRespondent27.70250.3221.0000.08chi-squareNone
Note: a Levels excluded: Insurance (life, property and casualty, health), n = 9, Commercial and business banking, n = 8, Payments and merchant acquiring (networks, acquiring, processing), n = 7, Credit card and payments issuing, n = 6, Retail and consumer banking, n = 9, Markets and trading (sales and trading, global markets), n = 8, Private equity and private capital, n = 5. b Levels excluded: Client servicing and communications, n = 8, Trade, settlement and reconciliation, n = 7, Payments, cards and disputes, n = 9, Credit, underwriting and suitability, n = 5, Regulatory, audit, tax and legal, n = 7, Financial crime and client onboarding, n = 5.

Segment families recorded at engagement level were tested on engagements, and respondent role and vendor type on respondents. After Holm correction, the source differed in no family.

Written Sentiment by Segment

Figure 3. Written sentiment by engagement status and lifecycle stage, share of each group's responsesExhibit
0%25%50%75%100%All responses (n = 37)48.6%10.8%37.8%Active, Build-Out (n = 11)54.5%45.5%Active, Transition/Cut-Over (n = 11)45.5%9.1%45.5%Active, Production (n = 4)1 of 41 of 41 of 41 of 4Canceled (n = 11)54.5%18.2%27.3%−10 Strongly negative−5 Negative0 Neutral+5 Positive+10 Strongly positive
Note: Each bar is the whole of its group's responses, divided by written score from −10 at the left to +10 at the right. Segments large enough to hold a label carry their share, or their count in groups of fewer than 10 responses.
Table 9. Written sentiment by segmentTable
SegmentResponsesRespondentsEngagementsMean written scoreNegativeStrongest negative (−10)
Engagement status and lifecycle stage
Active, Build-Out111110−0.56 of 11 (54.55%)0 of 11 (0.00%)
Active, Transition/Cut-Over111190.05 of 11 (45.45%)0 of 11 (0.00%)
Active, Production444‡2 of 41 of 4
Canceled111110−1.46 of 11 (54.55%)0 of 11 (0.00%)
Respondent role
Individual Contributor373733−0.819 of 37 (51.35%)1 of 37 (2.70%)
Engagement Manager000No responses
Portfolio Director000No responses
Customer Management Executive000No responses
Note: ‡ Base under 10. No statistic is reported.

Written sentiment did not differ by engagement status and lifecycle stage (Kruskal-Wallis H 0.66, df 3, p = 0.883, n = 33 engagements). All 37 responses came from individual contributors, so no test across roles was run.

Association with Cancellation Rate

Cancellation rate is the share of engagements canceled by the study date, measured at engagement level (150 engagements, 56 canceled).

Table 10. Association with cancellation rate by answer levelTable
Answer levelEngagementsCanceledCancellation rate
Vendor and partner training17529.41%
On the job16850.00%
Online courses12541.67%
University coursework15746.67%
Self-taught201050.00%
Employer training program14535.71%
Table 11. Association with cancellation rate, tests across answer levels by segmentTable
SegmentEngagementsChi-squaredfpHolm pCramer's VTest for p
Study set942.3950.7920.7920.16chi-square
Outsourcing model
CXM26a
ITO25a
BPO21a
KPO22a
Institution type
Asset management17a
Wealth management and private banking12a
Investment banking (advisory and capital-markets origination)13a
Insurance (life, property and casualty, health)9‡
Commercial and business banking8‡
Payments and merchant acquiring (networks, acquiring, processing)7‡
Credit card and payments issuing6‡
Retail and consumer banking9‡
Markets and trading (sales and trading, global markets)8‡
Private equity and private capital5‡
Process family
Corporate and back-office functions32a
Technology engineering and operations21a
Client servicing and communications8‡
Trade, settlement and reconciliation7‡
Payments, cards and disputes9‡
Credit, underwriting and suitability5‡
Regulatory, audit, tax and legal7‡
Financial crime and client onboarding5‡
Class of work
LOB Operational220.7310.3920.7840.18chi-square
Corporate Functional42a
Engagement status and lifecycle stage
Active, Build-Outb
Active, Transition/Cut-Overb
Active, Productionb
Canceledb
Note: ‡ Base under 10. No statistic is reported. a Not tested: fewer than two levels of 10 or more engagements. b Not tested: no variation in cancellation.

Across the 94 engagements with an assigned source, cancellation did not differ by source (chi-square 2.39, df = 5, p = 0.792, Cramer's V 0.16).

Association with Customer Satisfaction

Customer satisfaction is the percentage recorded against each engagement. The engagement-level estimate uses the active engagements in the sources with 10 or more active engagements.

Table 12. Association with customer satisfaction by answer levelTable
Answer levelEngagementsMean of the measureSD
Vendor and partner training1260.013.9
On the job8‡
Online courses7‡
University coursework8‡
Self-taught1063.914.0
Employer training program9‡
Note: ‡ Base under 10. No statistic is reported.
Table 13. Association with customer satisfaction, tests across answer levels by segmentTable
SegmentEngagementsWelch FdfpHolm pOmega squared90% CI
Study set220.431, 19.20.5210.521−0.027†
Outsourcing model
CXM15a
ITO14a
BPO12a
KPO13a
Institution type
Asset management9‡
Wealth management and private banking8‡
Investment banking (advisory and capital-markets origination)5‡
Insurance (life, property and casualty, health)6‡
Commercial and business banking5‡
Payments and merchant acquiring (networks, acquiring, processing)4‡
Credit card and payments issuing3‡
Retail and consumer banking5‡
Markets and trading (sales and trading, global markets)5‡
Private equity and private capital4‡
Process family
Corporate and back-office functions17a
Technology engineering and operations14a
Client servicing and communications3‡
Trade, settlement and reconciliation3‡
Payments, cards and disputes7‡
Credit, underwriting and suitability4‡
Regulatory, audit, tax and legal3‡
Financial crime and client onboarding3‡
Class of work
LOB Operational31a
Corporate Functional23a
Engagement status and lifecycle stage
Active, Build-Out13a
Active, Transition/Cut-Over30a
Active, Production11a
Canceledb
Note: † Base under 30. No interval is reported. ‡ Base under 10. No statistic is reported. a Not tested: fewer than two levels of 10 or more engagements. b Not tested: canceled engagements fall outside the active population.

Customer satisfaction did not differ by source (Welch F 0.43, p = 0.521, n = 22 engagements), on a base of 22 active engagements in the sources with 10 or more active engagements.

Association with the Engagement Indicator Mean

The engagement indicator mean is the mean of an engagement's ratings across the rated indicators, measured at engagement level (150 engagements). The source is not a rated indicator and does not enter the mean.

Table 14. Association with the engagement indicator mean by answer levelTable
Answer levelEngagementsMean of the measureSD
Vendor and partner training173.81.6
On the job163.91.6
Online courses123.91.6
University coursework154.01.2
Self-taught203.71.5
Employer training program144.31.6
Table 15. Association with the engagement indicator mean, tests across answer levels by segmentTable
SegmentEngagementsWelch FdfpHolm pOmega squared90% CI
Study set940.245, 39.30.9401.000−0.0400.000 to 0.010
Outsourcing model
CXM26a
ITO25a
BPO21a
KPO22a
Institution type
Asset management17a
Wealth management and private banking12a
Investment banking (advisory and capital-markets origination)13a
Insurance (life, property and casualty, health)9‡
Commercial and business banking8‡
Payments and merchant acquiring (networks, acquiring, processing)7‡
Credit card and payments issuing6‡
Retail and consumer banking9‡
Markets and trading (sales and trading, global markets)8‡
Private equity and private capital5‡
Process family
Corporate and back-office functions32a
Technology engineering and operations21a
Client servicing and communications8‡
Trade, settlement and reconciliation7‡
Payments, cards and disputes9‡
Credit, underwriting and suitability5‡
Regulatory, audit, tax and legal7‡
Financial crime and client onboarding5‡
Class of work
LOB Operational220.031, 19.50.8651.000−0.046†
Corporate Functional42a
Engagement status and lifecycle stage
Active, Build-Out13a
Active, Transition/Cut-Over30a
Active, Production11a
Canceled40a
Note: ‡ Base under 10. No statistic is reported. † Base under 30. No interval is reported. a Not tested: fewer than two levels of 10 or more engagements.

The engagement indicator mean did not differ by source (Welch F 0.24, p = 0.940, n = 94 engagements).

Association with Intent to Leave

Intent to leave is the answer to C019, “How likely are you to leave your current job in the next 12 months?”, rated 1 to 10 by up to 798 respondents. For the indicator, the engagement-level comparison uses the mean intent to leave of each engagement.

Table 16. Association with intent to leave by answer levelTable
Answer levelEngagementsMean of the measureSD
Vendor and partner training176.82.3
On the job167.42.5
Online courses127.02.2
University coursework157.22.2
Self-taught207.32.6
Employer training program146.32.4
Table 17. Association with intent to leave, tests across answer levels by segmentTable
SegmentEngagementsWelch FdfpHolm pOmega squared90% CI
Study set940.365, 39.70.8711.000−0.0330.000 to 0.031
Outsourcing model
CXM26a
ITO25a
BPO21a
KPO22a
Institution type
Asset management17a
Wealth management and private banking12a
Investment banking (advisory and capital-markets origination)13a
Insurance (life, property and casualty, health)9‡
Commercial and business banking8‡
Payments and merchant acquiring (networks, acquiring, processing)7‡
Credit card and payments issuing6‡
Retail and consumer banking9‡
Markets and trading (sales and trading, global markets)8‡
Private equity and private capital5‡
Process family
Corporate and back-office functions32a
Technology engineering and operations21a
Client servicing and communications8‡
Trade, settlement and reconciliation7‡
Payments, cards and disputes9‡
Credit, underwriting and suitability5‡
Regulatory, audit, tax and legal7‡
Financial crime and client onboarding5‡
Class of work
LOB Operational220.101, 19.60.7601.000−0.043†
Corporate Functional42a
Engagement status and lifecycle stage
Active, Build-Out13a
Active, Transition/Cut-Over30a
Active, Production11a
Canceled40a
Note: ‡ Base under 10. No statistic is reported. † Base under 30. No interval is reported. a Not tested: fewer than two levels of 10 or more engagements.

Intent to leave did not differ by source (Welch F 0.36, p = 0.871, n = 94 engagements).

Association of Written Sentiment with the Study Measures

Table 18. Written sentiment and the study measuresTable
ComparisonWith a negative responseWithout a negative responseTestStatisticnp
Cancellation rate, engagements5 of 17 (29.41%)51 of 133 (38.35%)Fisher exact1500.598
Customer satisfaction, active engagements63.33%, n = 1264.21%, n = 82Welch t−0.26940.797
Engagement mean written score and cancellation, engagementsPoint-biserial r−0.03330.854
Respondent mean written score and intent to leave, respondentsSpearman rho−0.13320.464

The comparisons set engagements with a negative written response against all other engagements, most of which had no written response addressing the indicator. Cancellation affected 5 of the 17 engagements with a negative response (29.41%), against 51 of the 133 other engagements (38.35%), a difference that did not reach significance at 0.05 (Fisher exact p = 0.598). Customer satisfaction was 63.33% on the 12 active engagements with a negative response, against 64.21% on the 82 other active engagements (Welch t −0.26, p = 0.797), a difference that did not reach significance at 0.05. The engagement mean written score against cancellation (r = −0.03, p = 0.854, n = 33 engagements) did not reach significance at 0.05, and the respondent mean written score against intent to leave (rho = −0.13, p = 0.464, n = 32 respondents) did not reach significance at 0.05. No written outcome test reached significance.

Comparison of Associations Across the Four Measures

Table 19. Associations across the four measuresTable
MeasureEffect sizenValuepEquivalent to zeroRank among the four
Cancellation rateOmega squared, 0 or 1 flag94−0.0300.792Yes2
Customer satisfactionOmega squared22−0.0270.521No1
Engagement indicator meanOmega squared94−0.0400.940Yes4
Intent to leaveOmega squared94−0.0330.871Yes3

Steiger's z compares two correlations, and a categorical indicator yields an effect size across its levels and not a correlation. The pairs of measures were therefore not compared. No association reached significance.

Clustering and Effective Sample Size

Table 20. Clustering and effective sample sizeTable
StatisticValueReading
Clusters (engagements)150Engagements with one or more answers.
Mean respondents per engagement5.3Answers per engagement, among engagements with one or more answers.
ICC(1)0.00Share of the variance that lies between engagements.
Design effect1.00One plus the product of the ICC and the mean cluster size less one.
Nominal respondent n800Every answer, with “I do not know” and “I prefer to not answer” counted as levels, as in the plurality rule.
Effective n800Nominal n divided by the design effect.
Cluster bootstrap appliedNoApplied to every respondent-level interval where the ICC is 0.05 or above.

Answers did not cluster within engagements (ICC 0.00): respondents on the same engagement named different sources. The design effect was 1.00, and the effective n equaled the 800 answers. The respondent-level tests used standard errors clustered by engagement.

Equivalence Tests of Null Results

Table 21. Equivalence tests of null resultsTable
MeasurenOmega squaredEquivalence boundEquivalence pEquivalent to zeroAchieved powerMDE at power 0.80
Cancellation rate94−0.030Omega squared < 0.0900.007Yes0.610.127
Customer satisfaction22−0.027Omega squared < 0.0900.187No0.290.283
Engagement indicator mean94−0.040Omega squared < 0.0900.002Yes0.610.127
Intent to leave94−0.033Omega squared < 0.0900.005Yes0.610.127

The engagement indicator mean (equivalence p = 0.002) and intent to leave (equivalence p = 0.005) were equivalent to zero, each on 94 engagements. Customer satisfaction was not (equivalence p = 0.187, n = 22 engagements), and its result is inconclusive rather than evidence of no association. The cancellation contrast carries no equivalence test.

Robustness Checks

Table 22. Robustness checksTable
CheckMeasurePrimary pCheck pCheck statisticConclusion holds
Kruskal-Wallis in place of Welch FCustomer satisfaction0.5210.6440.21Yes
Kruskal-Wallis in place of Welch FEngagement indicator mean0.9400.8971.63Yes
Kruskal-Wallis in place of Welch FIntent to leave0.8710.7642.58Yes
Active engagements onlyEngagement indicator mean0.9400.5950.29Yes
Active engagements onlyIntent to leave0.8710.6220.25Yes
Respondent level, cluster-robust Wald F by engagementCancellation rate0.7920.7840.49Yes
Respondent level, cluster-robust Wald F by engagementCustomer satisfaction0.5210.8090.46Yes
Respondent level, cluster-robust Wald F by engagementEngagement indicator mean0.9400.7410.55Yes
Respondent level, cluster-robust Wald F by engagementIntent to leave0.8710.6470.67Yes

All nine checks held. A check holds when its test falls on the same side of 0.05 as the primary test. The respondent-level test, with standard errors clustered by engagement, found no difference for any of the four measures (p from 0.647 to 0.809).

Measure Independence and Part-Whole Correction

Each of the four measures was computed from data the indicator does not contain. Each result is therefore a finding and not an arithmetic consequence.

Comparison with the Study Norm

A categorical indicator has no signed correlation with the study measures and therefore no profile against the study norm.

Comparison of Written Sentiment with Other Part 4 Indicators

Figure 4. Share of negative written responses, indicators of Part 4 with 30 or more written responses (n = 11 indicators)Exhibit
0%25%50%75%100%C021C020C018C092C017C012C071C080C014C069C084Share of negative responsesMedian of the 11 indicators
Note: Each column is one indicator, ordered from the highest share of negative responses. The indicator is the amber column. The dashed line marks the median share, 33.33%.

Among the 11 indicators of Part 4 with 30 or more written responses, the indicator's share of negative responses, 51.35%, ranked 5 of 11 from the highest, against a median of 33.33%. Its mean written score (−0.8, n = 37 responses) ranked 5 of 11 from the most negative, against a median of 0.8 across the 11 indicators.

Co-Occurring Indicators and Themes

Figure 5. Other indicators and themes in the same written responses (n = 37 responses)Exhibit
05.8811.7617.6423.52C12321 (56.76%)C12410 (27.03%)Theme: Skills learned9 (24.32%)Theme: Team or colleagues6 (16.22%)C0805 (13.51%)C0185 (13.51%)Theme: Positive with an embeddednegative5 (13.51%)C1574 (10.81%)C1254 (10.81%)C1224 (10.81%)C0144 (10.81%)Theme: The scale or question does notfit3 (8.11%)Theme: Respondent proposes a remedy2 (5.41%)Other indicatorsThemes
Note: Themes carry no score. Labels give the count and the share of the 37 responses. The figure shows each indicator and each theme among the five most frequent of its kind, with every tie at the cut. The questions of the indicators shown are quoted in the text where they lead. Emotional states in the same responses, as counts: angry 1, calm or fine 1, tired 1.

The indicator most often present in the same responses was C123, on the adequacy of the employer's investment in training its workforce, in 21 of the 37 responses (56.76%), followed by C124, on how well the employer's training programs build practical agentic AI skills, in 10. The most frequent theme was “Skills learned” (9 responses), every one a response to C030.

Interpretation of the Data

Respondents named six sources of their AI knowledge in almost equal numbers, and the source did not separate engagements on cancellation, customer satisfaction, the engagement indicator mean or intent to leave. Two of these results were equivalent to zero, and every robustness check held.

The written responses describe a different picture from the answer itself. Individual contributors who wrote said they learned the agents on live work, from colleagues and from outside material, after training of slides and videos, and that often no one above them knew the agents. The answers record where knowledge was said to come from, and the written responses record how thin the formal route was for those who wrote.

Limitations

The evidence is the reported experience of the vendor's delivery teams, collected in a single cross-section. The associations reported here are not causal.

The respondent gave one source. Most people learn from several, so the answer records the source the respondent judged most important.

Customer satisfaction was tested on 22 active engagements, a small base.

Shares of written sentiment are shares of responses and not of the study set of respondents. The written score takes five fixed values, so its mean and median are coarse. Responses clustered within engagements (37 responses on 33 engagements), and intervals for written sentiment therefore used a bootstrap by engagement. In the sample read by a second reader, the two readers agreed on the presence of the indicator in 5 of the 5 responses either reader marked, and on the score in 2 of the 3 responses both readers scored.

Definitions

Table 23. DefinitionsTable
TermDefinition
Response scaleThe answer is one of the listed sources of AI knowledge shown in the indicator specification. The indicator is not a rating. No distressed range applies.
Answering baseRespondents who gave one of the listed answers.
Non-answers“I do not know” and “I prefer to not answer” are counted separately. They are excluded from the answering base and from the shares.
Indicator classRespondent level: a property of the respondent rather than of the engagement.
Failure categoryThe five failure categories of the Study Taxonomy apply only to the rated indicators. The indicator carries none.
Assigned answerThe answer given by the largest number of an engagement's respondents. A tied engagement carries no assigned answer.
Written responseA written answer about the respondent's own engagement that addresses the indicator. Non-answers and answers about another engagement are excluded.
Primary and secondaryA primary response answered the written-answer question of the indicator's survey section. A secondary response answered a question in another section and addressed the indicator in its content.
Written sentiment scorea
Clustered testA test whose standard error allows for the answers of respondents on the same engagement to be alike.
Cluster bootstrapAn interval computed by resampling engagements rather than responses, so that the responses from one engagement stay together.
Note: a Each written response carries one of five scores: −10, −5, neutral, +5 or +10. The sign records whether the response described an effect on the engagement or on the respondent as harmful or helpful. The magnitude records its strength, and −10 marks a statement of totality, constancy or serious harm. A source of knowledge has no favorable or unfavorable value. Its score therefore records the effect the respondent associated with the way the respondent learned.

Sources and Provenance

Table 24. Sources and provenanceTable
SourceValue
Study dataStudy Data.xlsx, Data sheet, column C017, sha256 219eba0457274d04
Statistics enginecompute_statistics.py, sha256 738319eef764602b
Statistics recordComputed Statistics.json, record C017, sha256 1ae50a3637fc9370
ComputedOctober 13, 2026, 04:57 UTC
TemplateIndicator Template v2, version 2.0 of 10 October 2026
Figures placed by script1,312
Written responsesQuotes Master.jsonl, sha256 27da38085e2b13fd
Written sentiment scoresQuote Coding Rulebook, version 0.6, with section 21
Written-response computationqa_stats.py, sha256 74a61f3a96169da9. wfacts.py, sha256 b85fae20adf812a1. build_indicator.py, sha256 c7b10c57927ea734
Written-response recordsC017.qa.json, sha256 d53aa9cdead03ab3. C017.wfacts.json, sha256 e7086be208210ba8. Hand-written text handcraft/C017.json, sha256 0fd7726fda3b5a32

Continue the report

Executive summary →
Overview

Executive Summary

Four findings from 150 engagements

Read →
Introduction

Introduction

Scope, evidence, and how the report is organized

Read →
Part 1

Engagement Outcomes

Poor results on every measure, and what separates them

Read →
Part 2

The Five Failure Categories

Where failure concentrates, category by category

Read →
Part 3

Governance, Oversight, and Control

Controls that existed in name only

Read →
Part 4

The Delivery Workforce

Early in their careers, dissatisfied, and many intending to leave

Read →
Part 5

Manual Alteration

How the delivery team altered the work to sustain it

Read →
Part 6

Executive Action Plan

Seven immediate actions and a first-year plan

Read →
Appendices

Appendices

Definitions, indicator inventory, method, and disclosures

Read →
Data

Study Data

Deep-dive analyses of the study indicators, published in stages

Read →