Rate the effectiveness of your employer’s process and leadership development programs

Ratings of the effectiveness of the employer’s process and leadership development programs (mean 3.7, median 4.0, n = 803 respondents) were associated with all four study measures. Engagements with higher ratings were canceled less often (r = −0.59, 95% CI −0.69 to −0.48, p < 0.001, n = 150 engagements). Higher ratings came with higher customer satisfaction on active engagements (r = 0.69, 95% CI 0.57 to 0.78, p < 0.001, n = 94 active engagements). Ratings rose with the engagement indicator mean, computed with this item left out (r = 0.85, 95% CI 0.80 to 0.89, p < 0.001, n = 150 engagements). Respondents who gave higher ratings reported lower intent to leave (r = −0.61, cluster bootstrap 95% CI −0.67 to −0.53, p < 0.001, n = 753 respondents). Ratings differed after Holm control by engagement status and lifecycle stage (Welch F 31.56, Holm p < 0.001, omega squared 0.342) and by respondent role (Welch F 7.48, Holm p 0.006, omega squared 0.014).

The indicator had 7 written responses coded to it, from 7 of 850 respondents on 7 of 150 engagements. Negative responses numbered 4 of the 7 scored responses. The strongest negative score, −10, was carried by 1 of the 7 scored responses. Every response was a primary response, written in answer to a question in the indicator’s own survey section. The rating of respondents who wrote was not compared with that of respondents who did not, because 7 respondents wrote, fewer than 10.

The study set mean was 3.7 on 803 answering respondents, with a median of 4.0. 393 answers, 48.94% of the 803, were at the distressed end. A distressed engagement mean was recorded in 64 of 150 engagements.

All 7 responses were primary responses.

The distribution was reported as counts only, because fewer than 10 responses carried a score. Of the 7 scored responses, 4 were negative, 1 carried the strongest negative score and 3 were positive.

Figures are respondent level. Means are on the goodness view. A level under 30 prints no interval and a level under 10 prints counts only.

Engagement families were tested on engagement means, and respondent role and vendor type were tested on respondents. A distressed p marked exact is a Fisher exact test for a two by two table, and one marked simulated is a Monte Carlo test run until its p is stable to the third decimal, both used where an expected count is under 5. Engagement status and lifecycle stage differed after Holm control (Welch F 31.56, Holm p < 0.001, tested on engagement means). In Descriptive Statistics by Segment, Active, Production had the highest mean, 4.7 (n = 94 respondents), and Canceled the lowest, 2.4 (n = 306 respondents). Respondent role differed after Holm control (Welch F 7.48, Holm p 0.006, tested on respondents). In Descriptive Statistics by Segment, Portfolio Director had the highest mean, 4.7 (n = 31 respondents), and Individual Contributor the lowest, 3.6 (n = 621 respondents). No difference held after Holm control for outsourcing model, institution type, process family, class of work and vendor type.

The share of engagements canceled on the study date, measured at engagement level (base 150 engagements, 56 canceled). The association is a descriptive contrast, inflated because canceled engagements were rated after cancellation, and is not predictive.

The study set r was −0.59 (95% CI −0.69 to −0.48, Holm p < 0.001, n = 150 engagements). Of the 20 tested segment levels, 11 held after Holm control.

The satisfaction score recorded against each engagement on a 100-point scale. The primary estimate uses the 94 active engagements. A descriptive contrast uses all 150 engagements.

The study set r was 0.69 (95% CI 0.57 to 0.78, Holm p < 0.001, n = 94 engagements). Of the 16 tested segment levels, 13 held after Holm control.

The mean of an engagement’s 100 indicator ratings, measured at engagement level (base 150 engagements). Where this indicator is one of the 100, the mean is recomputed without it.

The study set r was 0.85 (95% CI 0.80 to 0.89, Holm p < 0.001, n = 150 engagements). Of the 24 tested segment levels, 24 held after Holm control.

The respondent’s stated likelihood of leaving the current job within 12 months, rated 1 to 10 (C019). Measured at respondent level on up to 798 respondents, with the engagement-level form reported alongside.

The study set r was −0.61 (95% CI −0.65 to −0.56, Holm p < 0.001, n = 753 respondents). Of the 37 tested segment levels, 36 held after Holm control.

Pairs were compared on the engagement level form of every measure, each turned so that a higher value was the better outcome, on the 150 engagements where the indicator and all four measures existed. Of the 6 pairs, 4 differed after Holm control.

Clustering reduced the 803 respondent answers to about 219 independent answers. Respondent level intervals therefore used the cluster bootstrap, and respondent level tests on intent to leave used cluster-robust errors.

No measure showed a tested null. No result was inconclusive. The detection floor column is the study floor at 150 engagements. Intent to leave was tested on its effective n.

Each change was computed before rounding, so it can differ by 0.01 from the difference of the two printed values.

Every check held. A check holds when its test lands on the same side of 0.05 as the primary test and, where the primary test is below 0.05, its r keeps the primary sign. Thin-base rule: engagements in any level under 30 engagements of outsourcing model or process family removed. Institution type is not used, because every one of its levels is under 30.

Each of the four measures was computed from data this indicator does not contain, so each result is a finding rather than an arithmetic consequence. This indicator is one of the 100 components, so the engagement indicator mean was recomputed without it, and the correction changed r by 0.00, from 0.86 to 0.85.

Each deviation was computed before rounding, so it can differ by 0.01 from the difference of the two printed values.

The comparison ran on 150 engagements. The indicator followed the study’s typical pattern on every measure.

The responses coded to the indicator were most often also coded to “Rate the factual reliability of the AI tools’ outputs on this engagement before any human review” (C157, 4 responses). The second-highest count was shared by “Rate the adequacy of your employer’s investment in training its workforce” (C123, 3 responses) and “Rate your employer’s readiness to take accountability when an agent it operates makes a costly error” (C130, 3 responses). The same responses carried 5 themes of the coding rulebook, each in 1 response.

Distressed answers made up less than half of the study set (393 of 803 answers, 48.94%). Active, Production had the highest mean by lifecycle stage (mean 4.7, n = 94 respondents), and Canceled had the lowest (mean 2.4, n = 306 respondents). Portfolio Director respondents gave the highest ratings by role (mean 4.7, n = 31 respondents), and Individual Contributor respondents gave the lowest (mean 3.6, n = 621 respondents). The associations held under every robustness check run on this item (22 of 22 checks).

The written responses were not set against the rating, because fewer than 10 responses carried a score. Written responses were voluntary. They recorded what the respondents who wrote reported, and how common it was is measured by the ratings. The analysis of the written responses rested on 7 responses, a base of anecdote strength.

The evidence is reported experience: what the delivery team said about its own engagement, gathered in one cross-section. The figures showed associations, not causes. Indicators about the financial institution were reported at second hand by vendor staff. Cancellation is a descriptive contrast, because canceled engagements were rated after they ended.

Tests across respondent role and vendor type were run at respondent level and treated respondents as independent. The section Clustering and Effective Sample Size states whether, and by how much, clustering within engagements reduced the information in respondent level figures.

Customer Management Executive answers were at anecdote strength (n = 1). Answers clustered within engagements, which reduced the effective number of independent answers (ICC 0.61, 803 answers worth about 219 independent answers).

Fewer than 10 responses carried a score, so written sentiment was reported as counts, with no statistic. Coding was performed by AI coders under a fixed rulebook. No response coded to the indicator fell in the sample second-coded under rulebook 0.5, so agreement on whether a response was coded to the indicator could not be assessed. No response coded to the indicator was second-scored under rulebook 0.5, so agreement on the score could not be assessed.