I or my management discussed process alteration with the client
Indicator C207 · By Phil Hatch · Akholi · October 9, 2026
On this page
25 sectionsIndicator Specification
| Field | Value |
|---|---|
| Indicator code | C207 |
| Question as asked | I or my management discussed process alteration with the client |
| Failure category | Not one of the 100 rated indicators |
| Place in the survey | Challenge Deep Dive > 5.6 Alterations > Client Processes |
| Indicator class | Respondent level |
| Indicator type | Yes or no |
| Response scale | Levels: Yes, No |
| Answering base | 414 of 850 respondents gave one of the listed answers. 473 answered, counting I do not know and I prefer to not answer, across 131 of 150 engagements |
| Non-answers | I do not know 37, I prefer to not answer 22, blank 377 |
| Written responses addressing the indicator | 124 responses from 114 respondents on 68 engagements (123 primary, 1 secondary) |
Summary of Findings
| Measure | Effect size | n | Value | p |
|---|---|---|---|---|
| Cancellation rate | Not tested: fewer than two levels of 10 or more engagements | |||
| Customer satisfaction | Not tested: fewer than two levels of 10 or more engagements | |||
| Engagement indicator mean | Not tested: fewer than two levels of 10 or more engagements | |||
| Intent to leave | Not tested: fewer than two levels of 10 or more engagements |
Of the 414 respondents who answered, 9 (2.17%) answered True and 405 (97.83%) answered False. No association with the study measures was tested, because fewer than two answers were assigned to at least ten engagements.
The share answering True did not differ by any segment family after Holm correction.
Of the 124 written responses, 99 were negative. At least 57 of the 99 negative responses said the financial institution was not told, or that the alteration stayed within the vendor team. Twelve responses said the financial institution had been told or already knew, and four described changes signed off by the financial institution before they were applied.
Distribution of Responses
| Answer | Respondents | Share of answered (base 414) | Cumulative share | Engagements with at least one | Share of 150 engagements | Distressed |
|---|---|---|---|---|---|---|
| Yes | 9 | 2.17% | 2.17% | 9 | 6.00% | Not applicable |
| No | 405 | 97.83% | 100.00% | 127 | 84.67% | Not applicable |
| I do not know | 37 | Not in base | Not in base | 15 | 10.00% | Not applicable |
| I prefer to not answer | 22 | Not in base | Not in base | 18 | 12.00% | Not applicable |
| Answered, total | 414 | 100.00% | 128 | 85.33% |
Of the 414 respondents who answered, 9 (2.17%) answered True and 405 (97.83%) answered False. The non-answers were I do not know (37) and I prefer to not answer (22). Each engagement was assigned the answer given by the largest number of its respondents. No engagement was assigned True, and False was the assigned answer on 116 engagements, “I do not know” on 7, “I prefer to not answer” on 1, and 26 engagements were tied or unanswered and carry no assigned answer.
Coverage and Origin of Written Responses
| Origin | Responses | Respondents | Engagements | Negative | Strongest negative (−10) | Positive | Mean written score |
|---|---|---|---|---|---|---|---|
| Primary | 123 | 113 | 67 | 98 of 123 (79.67%) | 73 of 123 (59.35%) | 15 of 123 (12.20%) | −6.2 |
| Secondary | 1 | 1 | 1 | 1 of 1 | 0 of 1 | 0 of 1 | ‡ |
| All responses | 124 | 114 | 68 | 99 of 124 (79.84%) | 73 of 124 (58.87%) | 15 of 124 (12.10%) | −6.2 |
The primary responses numbered 123, written in answer to C210, C208 and C204, the written-answer questions of the indicator's survey section. The remaining 1 responses, 0.81% of the total, were secondary and came from 1 other questions. That secondary share ranked 10th from the highest among the 13 Part 5 indicators with 30 or more written responses, against a median of 2.00%. Respondents who wrote both a primary and a secondary response numbered 0. Negative responses made up 79.67% of primary responses (98 of 123) and 1 of the 1 secondary responses.
In 57 of the 99 negative responses, the respondent said the financial institution was not told, or that the alteration stayed within the team. In 53, the team lead or manager was told or knew. Several respondents said the financial institution saw only the finished, clean version of the work.
Twelve responses said the financial institution had been told directly, had been looped in as the team built its own steps, or already knew that the team made process calls on its behalf. Four positive responses described every change routed through a joint forum or a formal review and signed off by the financial institution before it was applied. Nine responses said the respondent did not know whether the financial institution was told.
Distribution of Written Sentiment
| Statistic | Value | Basis |
|---|---|---|
| Mean written score | −6.2 | 95% CI −7.3 to −5.0, cluster bootstrap by engagement, 10,000 draws, n = 124 responses |
| Median written score | −10.0 | n = 124 responses |
| Negative responses | 99 of 124 (79.84%) | 95% CI 70.48% to 87.97% of 124 responses, cluster bootstrap by engagement |
| Strongest negative responses (−10) | 73 of 124 (58.87%) | Statements of totality, constancy or serious harm |
| Positive responses | 15 of 124 (12.10%) | +5 or +10 |
| Net balance | −67.74 percentage points | Share positive minus share negative, base 124 responses |
| Respondent-level mean | −6.3 | Mean of each respondent's own mean written score, n = 114 respondents |
Of the 124 responses, 99 (79.84%, 95% CI 70.48% to 87.97%) were negative and 73 (58.87%) carried the strongest negative score of −10. Positive responses numbered 15 (12.10%), and 10 were neutral. The mean written score was −6.2 (95% CI −7.3 to −5.0, n = 124 responses), and the median was −10.0.
Descriptive Statistics by Segment
| Segment | Base | Answered | Answer rate | Engagements | Yes, share | No, share |
|---|---|---|---|---|---|---|
| Study set | 850 | 414 | 48.71% of 850 | 131 | 2.17% | 97.83% |
| Outsourcing model | ||||||
| CXM | 232 | 107 | 46.12% of 232 | 36 | 1.87% | 98.13% |
| ITO | 244 | 118 | 48.36% of 244 | 33 | 0.85% | 99.15% |
| BPO | 209 | 110 | 52.63% of 209 | 36 | 2.73% | 97.27% |
| KPO | 165 | 79 | 47.88% of 165 | 26 | 3.80% | 96.20% |
| Institution type | ||||||
| Asset management | 132 | 56 | 42.42% of 132 | 19 | 3.57% | 96.43% |
| Wealth management and private banking | 140 | 76 | 54.29% of 140 | 19 | 1.32% | 98.68% |
| Investment banking (advisory and capital-markets origination) | 105 | 70 | 66.67% of 105 | 19 | 4.29% | 95.71% |
| Insurance (life, property and casualty, health) | 78 | 32 | 41.03% of 78 | 11 | 3.13% | 96.88% |
| Commercial and business banking | 82 | 34 | 41.46% of 82 | 11 | 0.00% | 100.00% |
| Payments and merchant acquiring (networks, acquiring, processing) | 71 | 33 | 46.48% of 71 | 11 | 0.00% | 100.00% |
| Credit card and payments issuing | 73 | 36 | 49.32% of 73 | 11 | 0.00% | 100.00% |
| Retail and consumer banking | 73 | 34 | 46.58% of 73 | 12 | 2.94% | 97.06% |
| Markets and trading (sales and trading, global markets) | 57 | 28 | 49.12% of 57 | 10 | 0.00% | 100.00% |
| Private equity and private capital | 39 | 15 | 38.46% of 39 | 8 | 6.67% | 93.33% |
| Process family | ||||||
| Corporate and back-office functions | 248 | 110 | 44.35% of 248 | 40 | 4.55% | 95.45% |
| Technology engineering and operations | 204 | 102 | 50.00% of 204 | 29 | 0.98% | 99.02% |
| Client servicing and communications | 94 | 49 | 52.13% of 94 | 14 | 0.00% | 100.00% |
| Trade, settlement and reconciliation | 80 | 43 | 53.75% of 80 | 15 | 2.33% | 97.67% |
| Payments, cards and disputes | 73 | 37 | 50.68% of 73 | 11 | 0.00% | 100.00% |
| Credit, underwriting and suitability | 57 | 26 | 45.61% of 57 | 7 | 0.00% | 100.00% |
| Regulatory, audit, tax and legal | 51 | 27 | 52.94% of 51 | 8 | 3.70% | 96.30% |
| Financial crime and client onboarding | 43 | 20 | 46.51% of 43 | 7 | 5.00% | 95.00% |
| Class of work | ||||||
| LOB Operational | 451 | 219 | 48.56% of 451 | 67 | 1.83% | 98.17% |
| Corporate Functional | 399 | 195 | 48.87% of 399 | 64 | 2.56% | 97.44% |
| Engagement status and lifecycle stage | ||||||
| Active, Build-Out | 175 | 72 | 41.14% of 175 | 26 | 0.00% | 100.00% |
| Active, Transition/Cut-Over | 250 | 113 | 45.20% of 250 | 35 | 1.77% | 98.23% |
| Active, Production | 103 | 38 | 36.89% of 103 | 16 | 0.00% | 100.00% |
| Canceled | 322 | 191 | 59.32% of 322 | 54 | 3.66% | 96.34% |
| Respondent role | ||||||
| Individual Contributor | 667 | 405 | 60.72% of 667 | 131 | 0.00% | 100.00% |
| Engagement Manager | 150 | 9 | 6.00% of 150 | 9 | 9 | 0 |
| Portfolio Director | 31 | 0 | 0.00% of 31 | 0 | 0 | 0 |
| Customer Management Executive | 2 | 0 | 0 of 2 | 0 | 0 | 0 |
| Vendor type | ||||||
| IT services major | 316 | 148 | 46.84% of 316 | 49 | 0.00% | 100.00% |
| BPO specialist | 198 | 104 | 52.53% of 198 | 36 | 3.85% | 96.15% |
| Consulting firm | 117 | 52 | 44.44% of 117 | 18 | 1.92% | 98.08% |
| Boutique or specialist provider | 87 | 42 | 48.28% of 87 | 15 | 4.76% | 95.24% |
| Pure-play agentic AI firm | 75 | 33 | 44.00% of 75 | 11 | 6.06% | 93.94% |
| Not reported, recovered by engagement | 57 | 35 | 61.40% of 57 | 34 | 0.00% | 100.00% |
Each share is computed on the respondents who answered in that row. Rows with fewer than 10 respondents report counts only.
Tests of Difference Between Segments
| Segment family | Unit | Chi-square | df | p | Holm p | Cramer's V | Test for p | Levels excluded |
|---|---|---|---|---|---|---|---|---|
| Outsourcing model | Engagement | a | None | |||||
| Institution type | Engagement | b | c | |||||
| Process family | Engagement | b | Credit, underwriting and suitability, n = 7, Regulatory, audit, tax and legal, n = 7, Financial crime and client onboarding, n = 5 | |||||
| Class of work | Engagement | a | None | |||||
| Engagement status and lifecycle stage | Engagement | a | None | |||||
| Respondent role | Respondent | b | Engagement Manager, n = 9, Portfolio Director, n = 0, Customer Management Executive, n = 0 | |||||
| Vendor type | Respondent | 9.12 | 5 | 0.106 | 0.106 | 0.15 | Monte Carlo, simulated | None |
Segment families recorded at engagement level were tested on engagements, and respondent role and vendor type on respondents. The share answering True did not differ by any segment family after Holm correction.
Written Sentiment by Segment
| Segment | Responses | Respondents | Engagements | Mean written score | Negative | Strongest negative (−10) |
|---|---|---|---|---|---|---|
| Engagement status and lifecycle stage | ||||||
| Active, Build-Out | 13 | 13 | 9 | −5.4 | 10 of 13 (76.92%) | 6 of 13 (46.15%) |
| Active, Transition/Cut-Over | 31 | 27 | 16 | −6.6 | 27 of 31 (87.10%) | 18 of 31 (58.06%) |
| Active, Production | 11 | 11 | 8 | −7.3 | 10 of 11 (90.91%) | 7 of 11 (63.64%) |
| Canceled | 69 | 63 | 35 | −5.9 | 52 of 69 (75.36%) | 42 of 69 (60.87%) |
| Respondent role | ||||||
| Individual Contributor | 112 | 104 | 63 | −7.2 | 96 of 112 (85.71%) | 72 of 112 (64.29%) |
| Engagement Manager | 7 | 5 | 5 | ‡ | 2 of 7 | 1 of 7 |
| Portfolio Director | 5 | 5 | 5 | ‡ | 1 of 5 | 0 of 5 |
| Customer Management Executive | 0 | 0 | 0 | No responses |
Written sentiment did not differ by engagement status and lifecycle stage across the 68 engagements with a written response (Kruskal-Wallis H 0.56, df 3, p = 0.906). Of the groups with 10 or more responses, responses from active engagements in Production were the most negative (mean −7.3, 11 responses) and responses from active engagements in Build-Out the least negative (mean −5.4, 13 responses).
No test across respondent roles was run, because fewer than two roles had 30 or more written responses. Individual contributors wrote 112 responses, engagement managers 7 and portfolio directors 5.
Association with Cancellation Rate
Cancellation rate is the share of engagements canceled by the study date, measured at engagement level (150 engagements, 56 canceled). Canceled engagements were rated after cancellation. The association is therefore a descriptive contrast and not a forecast of cancellation. Each test compares the engagements by their assigned answer, within the study set and within each segment.
| Answer level | Engagements | Canceled | Cancellation rate |
|---|---|---|---|
| Yes | 0 | No engagements | |
| No | 116 | 47 | 40.52% |
| I do not know | 7 | 4 | ‡ |
| I prefer to not answer | 1 | 1 | ‡ |
| Segment | Engagements | Chi-square | df | p | Holm p | Cramer's V | Test for p |
|---|---|---|---|---|---|---|---|
| Study set | 124 | a | |||||
| Outsourcing model | |||||||
| CXM | 33 | a | |||||
| ITO | 32 | a | |||||
| BPO | 34 | a | |||||
| KPO | 25 | a | |||||
| Institution type | |||||||
| Asset management | 18 | a | |||||
| Wealth management and private banking | 18 | a | |||||
| Investment banking (advisory and capital-markets origination) | 19 | a | |||||
| Insurance (life, property and casualty, health) | 10 | a | |||||
| Commercial and business banking | 10 | a | |||||
| Payments and merchant acquiring (networks, acquiring, processing) | 11 | a | |||||
| Credit card and payments issuing | 11 | a | |||||
| Retail and consumer banking | 11 | a | |||||
| Markets and trading (sales and trading, global markets) | 9 | ‡ | |||||
| Private equity and private capital | 7 | ‡ | |||||
| Process family | |||||||
| Corporate and back-office functions | 38 | a | |||||
| Technology engineering and operations | 28 | a | |||||
| Client servicing and communications | 12 | a | |||||
| Trade, settlement and reconciliation | 14 | a | |||||
| Payments, cards and disputes | 11 | a | |||||
| Credit, underwriting and suitability | 7 | ‡ | |||||
| Regulatory, audit, tax and legal | 8 | ‡ | |||||
| Financial crime and client onboarding | 6 | ‡ | |||||
| Class of work | |||||||
| LOB Operational | 62 | a | |||||
| Corporate Functional | 62 | a | |||||
| Engagement status and lifecycle stage | |||||||
| Active, Build-Out | b | ||||||
| Active, Transition/Cut-Over | b | ||||||
| Active, Production | b | ||||||
| Canceled | b |
Of the engagements assigned False, 47 of 116 were canceled. No test was computed, because fewer than two answers were assigned to at least ten engagements.
Association with Customer Satisfaction
Customer satisfaction is the percentage recorded against each engagement. The primary estimate uses the 94 active engagements.
| Answer level | Engagements | Mean of the measure | SD |
|---|---|---|---|
| Yes | 0 | No engagements | |
| No | 69 | 61.0 | 10.5 |
| I do not know | 3 | ‡ | |
| I prefer to not answer | 0 | No engagements |
| Segment | Engagements | Welch F | df | p | Holm p | Omega squared | 90% CI |
|---|---|---|---|---|---|---|---|
| Study set | 72 | a | |||||
| Outsourcing model | |||||||
| CXM | 18 | a | |||||
| ITO | 19 | a | |||||
| BPO | 21 | a | |||||
| KPO | 14 | a | |||||
| Institution type | |||||||
| Asset management | 6 | ‡ | |||||
| Wealth management and private banking | 12 | a | |||||
| Investment banking (advisory and capital-markets origination) | 10 | a | |||||
| Insurance (life, property and casualty, health) | 5 | ‡ | |||||
| Commercial and business banking | 6 | ‡ | |||||
| Payments and merchant acquiring (networks, acquiring, processing) | 8 | ‡ | |||||
| Credit card and payments issuing | 6 | ‡ | |||||
| Retail and consumer banking | 8 | ‡ | |||||
| Markets and trading (sales and trading, global markets) | 6 | ‡ | |||||
| Private equity and private capital | 5 | ‡ | |||||
| Process family | |||||||
| Corporate and back-office functions | 20 | a | |||||
| Technology engineering and operations | 19 | a | |||||
| Client servicing and communications | 3 | ‡ | |||||
| Trade, settlement and reconciliation | 7 | ‡ | |||||
| Payments, cards and disputes | 9 | ‡ | |||||
| Credit, underwriting and suitability | 6 | ‡ | |||||
| Regulatory, audit, tax and legal | 5 | ‡ | |||||
| Financial crime and client onboarding | 3 | ‡ | |||||
| Class of work | |||||||
| LOB Operational | 36 | a | |||||
| Corporate Functional | 36 | a | |||||
| Engagement status and lifecycle stage | |||||||
| Active, Build-Out | 25 | a | |||||
| Active, Transition/Cut-Over | 32 | a | |||||
| Active, Production | 15 | a | |||||
| Canceled | b |
Among active engagements, customer satisfaction was 61.0% (n = 69 engagements) where respondents most often answered False. No test was computed, because fewer than two answers were assigned to at least ten engagements.
Association with the Engagement Indicator Mean
The engagement indicator mean is the mean of an engagement's ratings across the rated indicators, measured at engagement level. The indicator is not a rated indicator and does not enter the mean.
| Answer level | Engagements | Mean of the measure | SD |
|---|---|---|---|
| Yes | 0 | No engagements | |
| No | 116 | 3.8 | 1.3 |
| I do not know | 7 | ‡ | |
| I prefer to not answer | 1 | ‡ |
| Segment | Engagements | Welch F | df | p | Holm p | Omega squared | 90% CI |
|---|---|---|---|---|---|---|---|
| Study set | 124 | a | |||||
| Outsourcing model | |||||||
| CXM | 33 | a | |||||
| ITO | 32 | a | |||||
| BPO | 34 | a | |||||
| KPO | 25 | a | |||||
| Institution type | |||||||
| Asset management | 18 | a | |||||
| Wealth management and private banking | 18 | a | |||||
| Investment banking (advisory and capital-markets origination) | 19 | a | |||||
| Insurance (life, property and casualty, health) | 10 | a | |||||
| Commercial and business banking | 10 | a | |||||
| Payments and merchant acquiring (networks, acquiring, processing) | 11 | a | |||||
| Credit card and payments issuing | 11 | a | |||||
| Retail and consumer banking | 11 | a | |||||
| Markets and trading (sales and trading, global markets) | 9 | ‡ | |||||
| Private equity and private capital | 7 | ‡ | |||||
| Process family | |||||||
| Corporate and back-office functions | 38 | a | |||||
| Technology engineering and operations | 28 | a | |||||
| Client servicing and communications | 12 | a | |||||
| Trade, settlement and reconciliation | 14 | a | |||||
| Payments, cards and disputes | 11 | a | |||||
| Credit, underwriting and suitability | 7 | ‡ | |||||
| Regulatory, audit, tax and legal | 8 | ‡ | |||||
| Financial crime and client onboarding | 6 | ‡ | |||||
| Class of work | |||||||
| LOB Operational | 62 | a | |||||
| Corporate Functional | 62 | a | |||||
| Engagement status and lifecycle stage | |||||||
| Active, Build-Out | 25 | a | |||||
| Active, Transition/Cut-Over | 32 | a | |||||
| Active, Production | 15 | a | |||||
| Canceled | 52 | a |
The engagement indicator mean was 3.8 (n = 116 engagements) where respondents most often answered False. No test was computed, because fewer than two answers were assigned to at least ten engagements.
Association with Intent to Leave
Intent to leave is the answer to C019, “How likely are you to leave your current job in the next 12 months?”, rated 1 to 10 by up to 798 respondents. For the indicator, the comparison uses the mean intent to leave of each engagement.
| Answer level | Engagements | Mean of the measure | SD |
|---|---|---|---|
| Yes | 0 | No engagements | |
| No | 116 | 7.1 | 2.2 |
| I do not know | 7 | ‡ | |
| I prefer to not answer | 1 | ‡ |
| Segment | Engagements | Welch F | df | p | Holm p | Omega squared | 90% CI |
|---|---|---|---|---|---|---|---|
| Study set | 124 | a | |||||
| Outsourcing model | |||||||
| CXM | 33 | a | |||||
| ITO | 32 | a | |||||
| BPO | 34 | a | |||||
| KPO | 25 | a | |||||
| Institution type | |||||||
| Asset management | 18 | a | |||||
| Wealth management and private banking | 18 | a | |||||
| Investment banking (advisory and capital-markets origination) | 19 | a | |||||
| Insurance (life, property and casualty, health) | 10 | a | |||||
| Commercial and business banking | 10 | a | |||||
| Payments and merchant acquiring (networks, acquiring, processing) | 11 | a | |||||
| Credit card and payments issuing | 11 | a | |||||
| Retail and consumer banking | 11 | a | |||||
| Markets and trading (sales and trading, global markets) | 9 | ‡ | |||||
| Private equity and private capital | 7 | ‡ | |||||
| Process family | |||||||
| Corporate and back-office functions | 38 | a | |||||
| Technology engineering and operations | 28 | a | |||||
| Client servicing and communications | 12 | a | |||||
| Trade, settlement and reconciliation | 14 | a | |||||
| Payments, cards and disputes | 11 | a | |||||
| Credit, underwriting and suitability | 7 | ‡ | |||||
| Regulatory, audit, tax and legal | 8 | ‡ | |||||
| Financial crime and client onboarding | 6 | ‡ | |||||
| Class of work | |||||||
| LOB Operational | 62 | a | |||||
| Corporate Functional | 62 | a | |||||
| Engagement status and lifecycle stage | |||||||
| Active, Build-Out | 25 | a | |||||
| Active, Transition/Cut-Over | 32 | a | |||||
| Active, Production | 15 | a | |||||
| Canceled | 52 | a |
The engagement mean of intent to leave was 7.1 (n = 116 engagements) where respondents most often answered False. No test was computed, because fewer than two answers were assigned to at least ten engagements.
Association of Written Sentiment with the Study Measures
| Comparison | With a negative response | Without a negative response | Test | Statistic | n | p |
|---|---|---|---|---|---|---|
| Cancellation rate, engagements | 28 of 58 (48.28%) | 28 of 92 (30.43%) | Fisher exact | 150 | 0.037 | |
| Customer satisfaction, active engagements | 60.90%, n = 30 | 65.59%, n = 64 | Welch t | −2.00 | 94 | < 0.050 |
| Engagement mean written score and cancellation, engagements | Point-biserial r | 0.07 | 68 | 0.565 | ||
| Respondent mean written score and intent to leave, respondents | Spearman rho | 0.00 | 104 | 0.975 |
Two comparisons set the engagements with at least one negative response against all other engagements. Of the 92 other engagements, 82 had no written response addressing the indicator. These two comparisons therefore do not isolate written sentiment. Of the 58 engagements with a negative response, 28 (48.28%) were canceled, against 28 of the 92 other engagements (30.43%). The difference reached significance (Fisher exact p = 0.037). Customer satisfaction among active engagements differed between the two groups (mean 60.90% on 30 engagements against 65.59% on 64, Welch t −2.00, p < 0.050).
Two correlations test written sentiment directly. Across the 68 engagements with a written response, the engagement mean written score was not associated with cancellation (point-biserial r 0.07, p = 0.565). Across 104 respondents, the respondent mean written score was not associated with intent to leave (Spearman rho 0.00, p = 0.975). Of the four comparisons, two reached significance at 0.05.
Comparison of Associations Across the Four Measures
| Measure | Effect size | n | Value | p | Equivalent to zero | Rank among the four |
|---|---|---|---|---|---|---|
| Cancellation rate | Not tested: fewer than two levels of 10 or more engagements | |||||
| Customer satisfaction | Not tested: fewer than two levels of 10 or more engagements | |||||
| Engagement indicator mean | Not tested: fewer than two levels of 10 or more engagements | |||||
| Intent to leave | Not tested: fewer than two levels of 10 or more engagements |
Fewer than two associations were tested, so no comparison across the measures applies.
Clustering and Effective Sample Size
| Statistic | Value | Reading |
|---|---|---|
| Clusters (engagements) | 131 | Engagements with one or more answers. |
| Mean respondents per engagement | 3.6 | Answers per engagement, among engagements with one or more answers. |
| ICC(1) | 0.38 | Share of the variance that lies between engagements. |
| Design effect | 2.00 | One plus the product of the ICC and the mean cluster size less one. |
| Nominal respondent n | 473 | Every answer, with “I do not know” and “I prefer to not answer” counted as levels, as in the plurality rule. |
| Effective n | 237 | Nominal n divided by the design effect. |
| Cluster bootstrap applied | Yes | Applied to every respondent-level interval where the ICC is 0.05 or above. |
Answers clustered within engagements (ICC 0.38). Clustering reduced the 473 respondent answers to the equivalent of about 237 independent answers. Respondent-level intervals therefore used the cluster bootstrap. Respondent-level tests used standard errors that allow for clustering by engagement.
Equivalence Tests of Null Results
| Measure | n | Omega squared | Equivalence bound | Equivalence p | Equivalent to zero | Achieved power | MDE at power 0.80 |
|---|---|---|---|---|---|---|---|
| Cancellation rate | a | ||||||
| Customer satisfaction | a | ||||||
| Engagement indicator mean | a | ||||||
| Intent to leave | a |
No association was tested, so no equivalence test applies. The equivalence bound is an omega squared of 0.090, the study's threshold for a negligible association expressed as variance explained.
Robustness Checks
| Check | Measure | Primary p | Check p | Check statistic | Conclusion holds |
|---|---|---|---|---|---|
| Kruskal-Wallis in place of Welch F | Customer satisfaction | Not computed | Not computed | Not computed | Not computed |
| Kruskal-Wallis in place of Welch F | Engagement indicator mean | Not computed | Not computed | Not computed | Not computed |
| Kruskal-Wallis in place of Welch F | Intent to leave | Not computed | Not computed | Not computed | Not computed |
| Active engagements only | Engagement indicator mean | Not computed | Not computed | Not computed | Not computed |
| Active engagements only | Intent to leave | Not computed | Not computed | Not computed | Not computed |
| Respondent level, cluster-robust Wald F by engagement | Cancellation rate | Not computed | 0.021 | 5.47 | Not computed |
| Respondent level, cluster-robust Wald F by engagement | Customer satisfaction | Not computed | < 0.001 | 16.19 | Not computed |
| Respondent level, cluster-robust Wald F by engagement | Engagement indicator mean | Not computed | < 0.001 | 27.38 | Not computed |
| Respondent level, cluster-robust Wald F by engagement | Intent to leave | Not computed | < 0.001 | 16.19 | Not computed |
No robustness check could be computed, because no primary test was computed.
Measure Independence and Part-Whole Correction
Each of the four measures was computed from data the indicator does not contain. Each result is therefore a finding and not an arithmetic consequence.
Comparison with the Study Norm
A categorical indicator has no signed correlation with the study measures and therefore no profile against the study norm. The comparison by answer appears in the four association sections.
Comparison of Written Sentiment with Other Part 5 Indicators
Among the 13 Part 5 indicators with 30 or more written responses, the indicator's share of negative responses, 79.84%, ranked 10th from the highest, against a median of 98.03%. Its mean written score, −6.2, ranked 9th from the most negative, against a median of −6.3.
Co-Occurring Indicators and Themes
The indicator most often present in the same responses was C206, “I discussed the process alteration with internal management”, in 94 of the 124 responses (75.81%). C205, “I have altered client processes” (78 responses), C209, “My greater team regularly alters client processes.” (27 responses), C109, “Rate how quickly the customer responds when this engagement raises issues that need customer action.” (18 responses) and C100, “Rate how completely the customer's processes are mapped and documented to the depth agentic AI requires.” (12 responses) followed.
The most frequent theme was “Told vendor-side management”, in 80 responses (64.52%). The second was “Financial institution was not told”, in 76 responses.
Interpretation of the Data
Of the respondents who answered, 2.17% said they or their management had discussed the process alteration with the financial institution (nine of 414). No engagement was assigned the answer True, so no association with the study measures could be tested.
The written evidence matches the answers. Respondents described processes altered as routine work, discussed within the vendor team and with the team lead, and seldom raised with the financial institution. The exceptions describe a financial institution told after the fact, or a joint forum that signed off each change before it was applied.
Limitations
The evidence is the reported experience of the vendor's delivery teams, collected in a single cross-section. The associations reported here are not causal. The answer is the respondent's own report of an alteration and not a record of it.
Each engagement carries the answer given by the largest number of its respondents, and tied engagements are excluded from the engagement-level tests.
Shares of written sentiment are shares of responses and not of the study set of respondents. The written score takes five fixed values, so its mean and median are coarse. Responses clustered within engagements (124 responses on 68 engagements). Intervals for written sentiment therefore used a bootstrap by engagement.
The written responses are counted by sets of words stated in each rule. A response that described a practice in other words is not counted, so each count is a floor.
With nine respondents answering True, the indicator supports a description of how rarely the process alteration was discussed and no comparison by answer.
Definitions
| Term | Definition |
|---|---|
| Response scale | The answer is True or False to the statement shown in the indicator specification. The indicator is not a rating. No distressed range applies. |
| Answering base | Respondents who answered True or False. |
| Non-answers | “I do not know” and “I prefer to not answer” are counted separately. They are excluded from the answering base and from the shares. |
| Indicator class | Respondent level: a report of the respondent's own action. Each engagement also carries the answer given by the largest number of its respondents, for the engagement-level tests. |
| Failure category | The five failure categories of the Study Taxonomy apply only to the rated indicators. The indicator carries none. |
| Engagement indicator mean | The mean of an engagement's ratings across the rated indicators. The indicator does not enter it. |
| Written response | A written answer about the respondent's own engagement that addresses the indicator. Non-answers and answers about another engagement are excluded. |
| Primary and secondary | A primary response answered the written-answer question of the indicator's survey section. A secondary response answered a question in another section and addressed the indicator in its content. |
| Written sentiment score | Each written response carries one of five scores: −10, −5, neutral, +5 or +10. The sign records whether the response described the state of the indicator as harmful or helpful to the engagement. The magnitude records its strength. A score of −10 marks a statement of totality, constancy or serious harm. |
| Cluster bootstrap | An interval computed by resampling engagements rather than responses, so that the responses from one engagement stay together. |
Sources and Provenance
| Source | Value |
|---|---|
| Study data | Study Data.xlsx, Data sheet, column C207, sha256 219eba0457274d04 |
| Statistics engine | compute_statistics.py, sha256 738319eef764602b |
| Statistics record | Computed Statistics.json, record C207, sha256 1ae50a3637fc9370 |
| Computed | October 13, 2026, 04:57 UTC |
| Template | Indicator Template v2, version 2.0 of 10 October 2026 |
| Figures placed by script | 781 |
| Written responses | Quotes Master.jsonl, sha256 27da38085e2b13fd |
| Written sentiment scores | Quote Coding Rulebook, version 0.6, with section 21 |
| Written-response computation | qa_stats.py, sha256 74a61f3a96169da9. wfacts.py, sha256 b85fae20adf812a1. build_indicator.py, sha256 c7b10c57927ea734 |
| Written-response records | C207.qa.json, sha256 dca5a9edbb2b8113. C207.wfacts.json, sha256 1d1dab0d93309509. Hand-written text handcraft/C207.json, sha256 ea6815b2e9f491ca |
