Study Data · Part 5 · Indicator Deep-Dive Analysis

The three alteration questions by respondent role: C199, C205, C211 by C005

Indicator ANALYSIS:alterations_by_role · By Phil Hatch · Akholi · October 9, 2026

Indicator Specification

Table 1. Indicator specificationTable
FieldValue
Indicator codeNone, a derived analysis
Question as askedThe three alteration questions by respondent role: C199, C205, C211 by C005
Failure categoryNot one of the 100 rated indicators
Place in the surveyNot a survey item
Indicator classDerived from other indicators
Indicator typeDerived analysis
Response scaleNot applicable, a derived analysis
Answering baseAs stated in Analysis Results
Non-answersNot applicable

Summary of Findings

Three Part 5 questions asked respondents whether they had altered data provided by the financial institution (C199), the financial institution's processes (C205) and AI outputs (C211). Saying True was almost entirely confined to individual contributors. Among engagement managers the share saying True was 6.08% or lower on each question, and no portfolio director said True to any of the three. The difference between roles reached significance on all three questions.

For C199, on data provided by the financial institution, 63.43% of individual contributors said True (n = 659), against 2.00% of engagement managers (n = 150) and 0.00% of portfolio directors (n = 30). The role difference gave chi-square 215.77 on two degrees of freedom, p < 0.001, n = 839 respondents.

For C205, on the financial institution's processes, 72.66% of individual contributors said True (n = 662), against 6.08% of engagement managers (n = 148) and 0.00% of portfolio directors (n = 31). The role difference gave chi-square 265.42 on two degrees of freedom, p < 0.001, n = 841 respondents.

For C211, on AI outputs, 42.46% of individual contributors said True (n = 650), against 1.36% of engagement managers (n = 147) and 0.00% of portfolio directors (n = 31). The role difference gave chi-square 107.09 on two degrees of freedom, p < 0.001, n = 828 respondents.

Analysis Results

The first table gives the base. The second gives the answers by role for each question, with the share saying True computed on the respondents who answered True or False. The third gives the test of the difference between roles.

Table 2. Analysis resultsTable
ResultValue
Unit of analysisRespondent
QuestionsC199, C205 and C211
Roles testedIndividual contributor, engagement manager, portfolio director
Role not in the testCustomer Management Executive, no True or False answer
Respondents answering True or False, C199839
Respondents answering True or False, C205841
Respondents answering True or False, C211828
Table 3. Answers by respondent roleTable
QuestionRoleTrueFalseAnswered True or FalseShare TrueI do not knowPrefer not to answer
C199Individual contributor41824165963.43%14
C199Engagement manager31471502.00%00
C199Portfolio director030300.00%01
C199Customer Management Executive000Not computed00
C199All roles42141883950.18%15
C205Individual contributor48118166272.66%04
C205Engagement manager91391486.08%02
C205Portfolio director031310.00%00
C205Customer Management Executive000Not computed00
C205All roles49035184158.26%06
C211Individual contributor27637465042.46%214
C211Engagement manager21451471.36%02
C211Portfolio director031310.00%00
C211Customer Management Executive000Not computed00
C211All roles27855082833.57%216
Table 4. Test of the difference between rolesTable
QuestionChi-squaredfpnRole not in the testIts respondentsIts True or False answers
C199215.772< 0.001839Customer Management Executive20
C205265.422< 0.001841Customer Management Executive20
C211107.092< 0.001828Customer Management Executive20

Distribution of Responses

Across all roles, 50.18% of the 839 respondents who answered True or False said True for C199, 58.26% of 841 for C205 and 33.57% of 828 for C211. The full distribution of each question, with its non-answers, is reported in the file of that indicator.

Descriptive Statistics by Segment

The analysis is itself a breakdown by respondent role, given in full in the second table. It was not broken down by any other segment.

Tests of Difference Between Segments

Each question was tested for a difference between individual contributors, engagement managers and portfolio directors by a chi-square test on the respondents who answered True or False. The respondents holding the role of Customer Management Executive gave no True or False answer to any of the three questions, so they appear as counts and are not in the test. All three tests reached significance.

Association with Cancellation Rate

Cancellation was not part of this analysis. The association of each question with cancellation is reported in the file of that indicator.

Association with Customer Satisfaction

Customer satisfaction was not part of this analysis. The association of each question with customer satisfaction is reported in the file of that indicator.

Association with the Engagement Indicator Mean

The engagement indicator mean was not part of this analysis.

Association with Intent to Leave

Intent to leave was not part of this analysis.

Comparison of Associations Across the Four Measures

The analysis concerns respondent role alone, so no comparison across the four measures applies.

Clustering and Effective Sample Size

The tests treat respondents as independent. Respondents on the same engagement share conditions, so the p-values are somewhat smaller than a test clustered by engagement would give. No clustered test was computed, so the p-values are approximate.

Equivalence Tests of Null Results

No equivalence test was computed, because every test found a difference.

Robustness Checks

No robustness checks were computed for this analysis.

Measure Independence and Part-Whole Correction

Respondent role and the three questions are separate fields, and neither enters the other. No part-whole correction applies.

Comparison with the Study Norm

A derived analysis has no profile against the study norm, so no comparison applies.

Interpretation of the Data

The alteration of data, processes and AI outputs was reported by the respondents who do the hands-on work. Individual contributors said True far more often than engagement managers, and portfolio directors never did. The pattern fits work done at the level where the data, the process and the output are handled, and it also fits a reluctance among managers to report alterations made under their oversight. The analysis cannot separate the two readings.

Limitations

The evidence is the reported experience of the vendor's delivery teams, collected in a single cross-section. The association reported here is not causal.

The questions ask whether the respondent personally altered data, processes or outputs. A manager who knew of alterations made by the team would answer False, so the role difference measures who made the alterations, not who knew.

The tests treat respondents as independent and were not clustered by engagement.

Definitions

Table 5. DefinitionsTable
TermDefinition
Share TrueThe respondents who answered True, divided by those who answered True or False.
Chi-square testA test of whether the share answering True differs between roles more than chance would allow.
Individual contributorA respondent who delivers the work directly, without managing others on the engagement.

Sources and Provenance

Table 6. Sources and provenanceTable
SourceValue
Study dataStudy Data.xlsx, sha256 219eba0457274d04
Statistics enginecompute_statistics.py, sha256 738319eef764602b
Statistics recordComputed Statistics.json, record ANALYSIS:alterations_by_role, sha256 1ae50a3637fc9370
ComputedOctober 13, 2026, 04:57 UTC
TemplateIndicator Template v2, version 2.0 of 10 October 2026
Figures placed by script172

Continue the report

Executive summary →
Overview

Executive Summary

Four findings from 150 engagements

Read →
Introduction

Introduction

Scope, evidence, and how the report is organized

Read →
Part 1

Engagement Outcomes

Poor results on every measure, and what separates them

Read →
Part 2

The Five Failure Categories

Where failure concentrates, category by category

Read →
Part 3

Governance, Oversight, and Control

Controls that existed in name only

Read →
Part 4

The Delivery Workforce

Early in their careers, dissatisfied, and many intending to leave

Read →
Part 5

Manual Alteration

How the delivery team altered the work to sustain it

Read →
Part 6

Executive Action Plan

Seven immediate actions and a first-year plan

Read →
Appendices

Appendices

Definitions, indicator inventory, method, and disclosures

Read →
Data

Study Data

Deep-dive analyses of the study indicators, published in stages

Read →