The engagement indicator mean summarizes ratings across the five indicator categories. This section examines its distribution, shared variation with the other core measures, and the extent to which individual indicators retain associations after adjustment for that mean.
The indicators should be interpreted as correlated assessments of engagement conditions. None of the 300 partial tests met the multiplicity-adjusted significance criterion. This limits claims about indicator-specific associations but does not prove that individual conditions have no independent effect.
The vendor's reporting rolled up successful completions across all payroll runs company-wide, which buried the specific business unit where repeated errors were occurring until the pattern was undeniable and the contract had already soured.”
INDIVIDUAL CONTRIBUTOR · BPO, Corporate and back-office functions
1 The Engagement Indicator Mean as a Summary Measure
The summary averages respondents' ratings across the 100 indicators and then aggregates them by engagement. Its construction and the leave-one-out procedure are defined in the Introduction, section 5.1 of “Study Design, Evidentiary Basis, and Definitions.”
Across the 150 engagements, the engagement indicator mean was 4.02 and the median 3.8, from 1.2 to 7.3. The rating behind it was given by all 850 respondents, each of whom rated at least one of the 100 indicators. Canceled engagements recorded a mean of 2.6 (n = 56 engagements), compared with 4.8 to 4.9 in the three active stages.
A higher engagement indicator mean was associated with less cancellation (r = −0.72, 95% CI −0.79 to −0.63, n = 150 engagements). It was associated with higher customer satisfaction among the active engagements (r = 0.84, 95% CI 0.77 to 0.89, n = 94 engagements). The mean was also associated with lower intent to leave (r = −0.78, cluster bootstrap 95% CI −0.82 to −0.74, n = 798 respondents). The associations were retained under the applicable checks in the 15-check measure protocol.
Honestly it comes down to the deadline being fixed while everything else moves. Model output shifts between runs, nobody on the client side is willing to own the call, and change control cannot keep pace, so I sign off on adjusted numbers myself rather than let the deadline slip again. I know it is not strictly how this is meant to work, and I have said so to both sides without anything changing.”
ENGAGEMENT MANAGER · KPO, Corporate and back-office functions
2 Shared Structure Across the Four Study Measures
The four core study measures shared substantial variation across engagements, and the other three measures differed systematically across the engagement indicator quintiles. Pairwise correlations and principal component analysis describe the shared structure. Principal component analysis identifies statistical combinations that capture shared variation among the measures. Figure 1.15 shows the observed cancellation gradient across those quintiles.
Scroll horizontally to view the complete figure.
Cancellation declined across the engagement indicator bands | ||||||||||||||||||||||||||||||||||||||||||||
Cancellation rate by engagement indicator quintileStudy set, 56 of 150 engagements (37.33%) Exact chart data
|
Note: The groups are ordered categories. The connecting lines do not represent elapsed time.
Canceled engagements: 28, 19, 9, 0, and 0 of 30 in bands 1 to 5. The bands differed in cancellation (chi-square 85.32, df = 4, p < 0.001, n = 150 engagements).
The measures share substantial variance. An indicator associated with several outcomes does not supply several independent confirmations of the same finding. Common respondent perceptions and engagement circumstances may contribute to the pattern.
2.1 A Single Dominant Component Across the Four Study Measures
The four core measures were correlated with one another in every pair across the 150 engagements. The absolute correlations ranged from 0.72 to 0.94. The strongest pair was cancellation with customer satisfaction (r = −0.94, 95% CI −0.96 to −0.92, p < 0.001, n = 150 engagements). The weakest was cancellation with the engagement indicator mean (r = −0.72, 95% CI −0.79 to −0.63, p < 0.001, n = 150 engagements).
Principal component analysis identified one dominant component, accounting for 88.23% of variance across the 150 engagements. An eigenvalue is the variance represented by a component. Its eigenvalue was 3.53, compared with 0.30 for the second component. A loading expresses how strongly a measure relates to a component. Absolute loadings on the first component were at least 0.91. Cancellation and intent to leave loaded negatively. The result describes shared variation, not a causal mechanism or the interchangeability of the four measures.
2.2 Differences in the Study Measures Across Indicator Quintiles
The 150 engagements were ranked by engagement indicator mean and divided into five bands of 30. No study measure was worse in any band than in the band before it.
Band | Engagements | Engagement indicator mean | Canceled | Cancellation rate | Customer satisfaction (mean) | Intent to leave (engagement mean) |
|---|---|---|---|---|---|---|
1, lowest | 30 | 2.2 | 28 | 93.33% | 13.0 | 9.1 |
2 | 30 | 3.0 | 19 | 63.33% | 24.3 | 8.5 |
3 | 30 | 3.8 | 9 | 30.00% | 43.0 | 7.0 |
4 | 30 | 4.9 | 0 | 0.00% | 63.2 | 5.4 |
5, highest | 30 | 6.2 | 0 | 0.00% | 75.8 | 3.8 |
The bands differed in cancellation (chi-square 85.32, df = 4, p < 0.001, n = 150 engagements). They differed in customer satisfaction (F = 84.65, df = 4 and 145, p < 0.001, omega squared 0.690, n = 150 engagements). The bands also differed in intent to leave (F = 76.29, df = 4 and 145, p < 0.001, omega squared 0.668, n = 150 engagements). Here F is the test statistic for differences in means, df denotes degrees of freedom, and omega squared estimates the proportion of variation associated with the groups. Degrees of freedom describe the information available to estimate variation. The rise of the engagement indicator mean from band to band follows from how the bands were built and is not a finding. The behavior of the other three measures is. Engagements in the lowest indicator quintile were almost all canceled, at 28 of 30. None of the 60 engagements in the two highest indicator quintiles had been canceled.
2.3 Persistence of the Associations Among Active Engagements
Among the 94 engagements not canceled by the study date, the three remaining core measures were correlated in every pair. Higher customer satisfaction was associated with a higher engagement indicator mean (r = 0.84, 95% CI 0.77 to 0.89, p < 0.001, n = 94 engagements). It was associated with lower intent to leave (r = −0.73, 95% CI −0.81 to −0.62, p < 0.001, n = 94 engagements). A higher engagement indicator mean was associated with lower intent to leave (r = −0.69, 95% CI −0.78 to −0.57, p < 0.001, n = 94 engagements).
Among active engagements, higher perceived cancellation likelihood was associated with lower customer satisfaction (r = −0.66, 95% CI −0.76 to −0.52, p < 0.001, n = 94). This is an additional subjective-risk measure. Realized cancellation is constant in the active group, so the active-only analysis supports associations among the three remaining core measures rather than reproducing the original four-measure structure.
3 Indicators Net of the Engagement Indicator Mean
Partial correlations examine whether an indicator retains an association after accounting for the engagement indicator mean. These tests inform interpretation of individual indicators. They do not identify causal effects or demonstrate that all conditions operate as a single mechanism.
Each of the 100 rated indicators was correlated with cancellation, customer satisfaction, and the engagement mean of intent to leave, with the engagement indicator mean held constant. The mean was recomputed without the indicator under test. In 33 of the 300 tests, p was under 0.05, against 15 expected by chance alone.
None of the 300 partial tests remained statistically significant after false-discovery-rate adjustment. The smallest adjusted q value was 0.323. The largest absolute partial correlation was 0.25. Its magnitude was below the study's 0.30 equivalence bound, but a point estimate alone does not establish equivalence.
Accordingly, the report does not attribute outcomes independently to an individual indicator. Differences in observed coefficients are descriptive unless a specified comparison test supports a stronger conclusion. The written accounts help distinguish the conditions respondents described without establishing their separate causal contributions.
To be fair the outside systems were not what killed this. The card platform worked, the statement print vendor did what they said. We just never got far enough to test any of it properly, two months and it was over. What broke was on our side and their side, the feeds we build ourselves. I would not put the outside people in the same bucket, they held up fine in the little we did.”
INDIVIDUAL CONTRIBUTOR · BPO, Payments, cards and disputes