What stage is the engagement in?

Engagement Canceled was the most common stage answer (309 of 810 respondents, 38.15%), followed by Development / buildout (131 of 810 respondents, 16.17%). Cancellation was an identity with the Engagement Canceled answer (by construction). Customer satisfaction on active engagements did not differ across the stage answers tested (Welch F 0.69, p 0.603, omega squared −0.013, n = 85 engagements), with Engagement Canceled, Team Buildout, and On Hold left out of the test (n = 0, n = 7, n = 2 engagements). The engagement indicator mean differed across stage answers (Welch F 40.50, p < 0.001, omega squared 0.534, n = 141 engagements), from Engagement Canceled at the low end (mean 2.6, n = 56 engagements) to Development / buildout at the high end (mean 5.2, n = 24 engagements). Intent to leave also differed across stage answers (Welch F 71.51, p < 0.001, omega squared 0.651, n = 141 engagements), from Engagement Canceled at the high end (mean 9.3, n = 56 engagements) to Planning at the low end (mean 4.9, n = 10 engagements). The answer mix differed after Holm control by engagement status and lifecycle stage (chi-square 450.00, Holm p < 0.001, Cramer’s V 1.00) and by vendor type (chi-square 166.88, Holm p < 0.001, Cramer’s V 0.20).

The indicator had 162 written responses coded to it, from 139 of 850 respondents on 69 of 150 engagements. Negative responses numbered 33 of the 162 scored responses (20.37%). The strongest negative score, −10, was carried by 10 of the 162 scored responses (6.17%). Every response was a secondary response, written in answer to a question outside the indicator’s own survey section.

A categorical answer has no engagement mean. Each engagement took its respondents’ plurality answer, and tied engagements were left out. An answer was assigned to 150 of 150 engagements.

Distressed is not applicable, because the answer has no good or bad end on a scale.

All 162 responses were secondary responses. The secondary responses came from 139 respondents on 69 engagements, in answer to 14 questions outside the indicator’s own survey section (most often C030 with 61 responses, C156 with 20 responses and C114 with 17 responses). Among the Part 1 indicators with 30 or more scored responses, the secondary share of the indicator ranked 1 of 6 from the highest, tied with 4 other indicators. The median secondary share was 100.00% across the 6 indicators.

The indicator has no good or bad end. Under the coding rulebook’s rule for such indicators, the written score records the effect each response states on the engagement or on the respondent. The survey question is “What stage is the engagement in?” The frequency, scope and consequence a response states set the strength of its score. A score of +10 marks a response that describes “this attribute of the engagement as helping the engagement at its best.” A score of −10 marks a response that describes “this attribute as harming the engagement at its worst . . . .”

Of the 162 scored responses, 33 (20.37%) were negative, 10 (6.17%) carried the strongest negative score and 5 (3.09%) were positive. The responses had a mean written score of −1.2 (95% CI −1.7 to −0.7, n = 162 responses on 69 engagements).

Shares are of the answered respondents in each row, which is the base beside them. A row under 10 prints counts in place of shares.

Engagement families were tested on engagement answers, and respondent role and vendor type were tested on respondents. Engagement status and lifecycle stage differed in the answer mix after Holm control (chi-square 450.00, Holm p < 0.001, Cramer’s V 1.00). Vendor type differed in the answer mix after Holm control (chi-square 166.88, Holm p < 0.001, Cramer’s V 0.20). No difference held after Holm control for outsourcing model, institution type, process family, class of work and respondent role.

Scored responses came from 11 engagements in the Active, Build-Out group, 9 engagements in the Active, Transition/Cut-Over group, 1 engagement in the Active, Production group and 48 engagements in the Canceled group. A test of mean written scores across the groups, with each engagement counted once, showed no difference that reached significance at 0.05 (Kruskal-Wallis H 3.05, df 2, p = 0.218, n = 68 engagements). The test compared the Active, Build-Out group, the Active, Transition/Cut-Over group and the Canceled group. It left out the Active, Production group, which held fewer than two engagements. Groups under 10 responses are reported as counts only. Negative responses numbered 0 of 1 in the Active, Production group.

The role test compared individual contributors and engagement managers, the roles with 30 or more scored responses. A test of mean written scores between the two roles, with each respondent counted once, showed no difference that reached significance at 0.05 (Kruskal-Wallis H 1.91, df 1, p = 0.167, n = 133 respondents). The roles compared had mean written scores of −1.3 for individual contributors (n = 109 responses) and −0.4 for engagement managers (n = 45 responses). Roles under 10 responses are reported as counts only. Among portfolio directors, negative responses numbered 4 of 7. Among customer management executives, negative responses numbered 0 of 1.

The share of engagements canceled on the study date, measured at engagement level (base 150 engagements, 56 canceled). The association is a descriptive contrast, inflated because canceled engagements were rated after cancellation, and is not predictive.

This answer is the measure by construction, so its association with the measure is an identity and not a finding.

The satisfaction score recorded against each engagement on a 100-point scale. The primary estimate uses the 94 active engagements. A descriptive contrast uses all 150 engagements.

The measure by the engagement’s assigned answer.

The test across answer levels, in the study set and inside each segment. Levels under 10 engagements were left out of each test.

The measure in the study set did not differ across answer levels (Welch F 0.69, p 0.603, n = 85 engagements). The clean estimate ran inside active engagements only.

The mean of an engagement’s 100 indicator ratings, measured at engagement level (base 150 engagements). Where this indicator is one of the 100, the mean is recomputed without it.

The measure by the engagement’s assigned answer.

The test across answer levels, in the study set and inside each segment. Levels under 10 engagements were left out of each test.

The measure in the study set differed across answer levels (Welch F 40.50, p < 0.001, omega squared 0.534, n = 141 engagements).

The respondent’s stated likelihood of leaving the current job within 12 months, rated 1 to 10 (C019). Measured at respondent level on up to 798 respondents, with the engagement-level form reported alongside.

The measure by the engagement’s assigned answer.

The test across answer levels, in the study set and inside each segment. Levels under 10 engagements were left out of each test.

The measure in the study set differed across answer levels (Welch F 71.51, p < 0.001, omega squared 0.651, n = 141 engagements).

The indicator is itself the study measure of cancellation, so written sentiment was not tested against cancellation. The customer satisfaction contrast compared engagements with a negative written response against all other engagements. The comparison group comprised 123 other engagements, of which 81 had no scored written response coded to the indicator and 42 had only scored responses that were not negative. The contrast therefore did not test written sentiment by itself. Mean customer satisfaction was 58.00% on the 10 active engagements with a negative response, against 64.82% on the 84 other active engagements. The difference did not reach significance at 0.05 (Welch t −1.74, p = 0.109). The association of written sentiment itself with the study measures was tested by one correlation. The association of the respondent mean written score with intent to leave did not reach significance at 0.05 (rho = 0.01, p = 0.877, n = 134 respondents). Neither of the two outcome comparisons reached significance at 0.05.

Steiger’s z compares two correlations, and a categorical answer yields an effect size across levels rather than a correlation, so the pairs are not tested.

Clustering reduced the 810 respondent answers to about 150 independent answers. Respondent level intervals therefore used the cluster bootstrap, and respondent level tests on intent to leave used cluster-robust errors.

The bound is the ruled |r| < 0.30 expressed as variance explained (V15-R22). Equivalent to zero: customer satisfaction.

Rows for cancellation rate are left out because one answer of this indicator is that measure by construction. Checks that did not hold: “Active engagements only” on engagement indicator mean, “Active engagements only” on intent to leave and “Respondent level, cluster-robust Wald F by engagement” on customer satisfaction. A check holds when its test lands on the same side of 0.05 as the primary test.

The answer Engagement Canceled marks exactly the respondents on canceled engagements, so the cancellation cell is an identity by construction.

Not applicable. A categorical item has no signed correlation, so it has no profile. Its per-level means carry the comparison.

The peer set comprised 101 indicators of the four Parts with 30 or more scored responses. The indicator was within the peer set. Its share of negative responses ranked 92 of 101 from the highest. Its mean written score (−1.2, n = 162 responses) ranked 89 of 101 from the most negative. The peer median of the mean written score was −4.6 across the 101 indicators.

The indicator’s twin, ENG:Engagement Phase, records the same fact under the coding rulebook’s twin rule. The twin was coded in all 162 responses coded to the indicator. The twin is left out of the counts that follow. The responses coded to the indicator were most often also coded to “ENG:Is Canceled?” (52 responses). The second most frequent co-coded indicator was “How long ago did this specific engagement start, in months?” (C035, 22 responses). The most frequent theme of the coding rulebook in the same responses was “Skills learned” (49 responses).

Respondents on canceled engagements reported a lower engagement indicator mean (mean 2.6, n = 56 engagements) than respondents at any active stage, including Production/Delivery (mean 4.8, n = 20 engagements). Intent to leave on canceled engagements (mean 9.3, n = 56 engagements) was higher than at any active stage, including Knowledge Transfer (mean 5.4, n = 14 engagements). Customer satisfaction did not differ across the active stages (Welch F 0.69, p 0.603, omega squared −0.013, n = 85 engagements).

The indicator is a survey question that is not a rating, so the written responses were not set against a rating. Written responses were voluntary. They recorded what the respondents who wrote reported, and how common it was is measured by the answers.

The evidence is reported experience: what the delivery team said about its own engagement, gathered in one cross-section. The figures showed associations, not causes. Indicators about the financial institution were reported at second hand by vendor staff. Cancellation is a descriptive contrast, because canceled engagements were rated after they ended.

Tests across respondent role and vendor type were run at respondent level and treated respondents as independent. The section Clustering and Effective Sample Size states whether, and by how much, clustering within engagements reduced the information in respondent level figures.

Team Buildout and On Hold were left out of the engagement indicator mean test (n = 7, n = 2 engagements) and the intent to leave test (n = 7, n = 2 engagements) because of small bases. The robustness checks held in part on this item (5 of 8 checks).

Shares of written sentiment were shares of responses, not of the study set of respondents. The written score took five fixed values, so its mean and median were coarse. Responses clustered within engagements. Intervals for written sentiment used a bootstrap by engagement. Coding was performed by AI coders under a fixed rulebook. Agreement on whether a response was coded to the indicator rested on 27 responses second-coded under rulebook 0.5, with the coders agreeing on 21 of 27. Agreement on the score rested on 11 responses second-scored under rulebook 0.6, with the scores agreeing in 4 of 11. These bases of second-coded responses are too small for an indicator-level alpha.