Customer Satisfaction, Delivery Performance, and Measured Return

Recorded customer satisfaction was substantially lower on canceled engagements, with no overlap between the canceled and active distributions. Section 1.1 reports their ranges. Both the recorded score and the separate perception rating were supplied by vendor personnel. Neither was verified against financial institution records.

Delivery-date performance, budget performance, and progress against key performance indicators received mean ratings between 4.8 and 4.9. These indicator-level results and their associations are reported in section 2. They describe perceived delivery performance rather than independently verified service outcomes.

“The problem is we raise the same funding cutoff issue on every status call and get nowhere, we learned the actual process just by watching their ops team work it live. And any decision that needs sign off stalls enough to blow the settlement window.” -Individual Contributor, BPO, Trade, settlement and reconciliation

“KPIs keep slipping and nobody agrees on why” -Individual Contributor, BPO, Payments, cards and disputes

“ROI case for this was shaky from the start” -Individual Contributor, KPO, Corporate and back-office functions

Return analysis is restricted to Production engagements and, for the reported correlations, to the 17 with positive recorded returns. The small selected sample and unspecified financial measurement conventions limit comparison and generalization (section 3).

1 Recorded and Perceived Customer Satisfaction

The study distinguishes a 100-point satisfaction score recorded for each engagement from a separate 10-point item asking whether the financial institution was satisfied with performance. Figure 1.9 presents the distribution of recorded scores.

1.1 Non-Overlapping Satisfaction of Canceled and Active Engagements

Recorded customer satisfaction averaged 43.85 points, with a median of 53.50, across 150 engagements. Active engagements averaged 64.10 points and ranged from 40.00 to 89.00 (n = 94). Canceled engagements averaged 9.88 points and ranged from 1.00 to 20.00 (n = 56). The two distributions did not overlap.

Satisfaction was associated with every other study measure, and the associations were retained under the applicable checks in the 16-check measure protocol. Its association with cancellation (r = −0.94, 95% CI −0.96 to −0.92, n = 150 engagements) measures the separation between the two groups. Higher satisfaction was associated with a higher engagement indicator mean (r = 0.85, 95% CI 0.80 to 0.89, n = 150 engagements). It was also associated with lower intent to leave (r = −0.84, cluster bootstrap 95% CI −0.87 to −0.81, n = 798 respondents).

The associations with the engagement indicator mean and with intent to leave did not depend on the canceled engagements. Among the 94 active engagements alone, they held at r = 0.84 and r = −0.63, respectively. Among the engagements still running, those delivered under better circumstances recorded higher satisfaction, and their delivery teams were less inclined to leave.

1.2 Delivery Team Perception of Financial Institution Satisfaction

Indicator C048 · The customer is happy with the performance of the engagement at this stage · Full statistical analysis ›

Individual contributors assessed financial institution satisfaction less favorably than portfolio directors, and ratings were lowest on canceled engagements. The mean rating was 5.0, and the median was 5.0 (n = 379 respondents). 244 respondents answered: “I do not know.”

Perceived financial institution satisfaction was associated with all four core measures and passed the applicable checks in the 22-check protocol. Its correlation with cancellation was −0.57 (95% CI −0.67 to −0.45, n = 150 engagements). Among active engagements, its correlation with recorded customer satisfaction was 0.53 (95% CI 0.36 to 0.66, n = 94 engagements). Its correlation with the engagement indicator mean was 0.74 (95% CI 0.65 to 0.80, n = 150 engagements), the largest observed coefficient among these comparisons. Coefficient ordering alone does not establish a difference in the underlying association strengths. Distressed ratings came from 89 of the 379 answers (23.48%), and the engagement mean was distressed on 29 of the 150 engagements.

The written responses that addressed the financial institution's satisfaction numbered 13, from 13 respondents on 12 engagements, and 9 of the 13 were negative.

“When the agent mis-cleared a set of cash breaks in March, our own side took two weeks to decide who owned the fix. The platform is good, the answer on accountability was slower than the client's patience.” -Engagement Manager, BPO, Trade, settlement and reconciliation

“…I sat between the client asking why supplier records were still wrong and my own management asking me for a green status. Both directions, every day. …” -Engagement Manager, CXM, Corporate and back-office functions

“Not everything here is same, main thing is templates get changed their side, we find out when client complains” -Individual Contributor, CXM, Client servicing and communications

2 Delivery Against Dates, Budget, and Key Performance Indicators

Reported delivery assessments were concentrated in a narrow range, with favorable-direction means from 4.8 to 4.9 (n = 554–798 respondents). The four questions addressed delivery dates, budget adherence, progress toward key performance indicators, and respondents' knowledge of those indicators. The measures and the practices described in written answers remain distinct subjects.

“sign off arrive after the filing deadline it was even meant for, the Review Board sit too rarely to help, so i decide myself and kindly hope it hold up later” -Individual Contributor, KPO, Regulatory, audit, tax and legal

2.1 Reported Meeting Delivery Dates

Indicator C049 · The engagement is meeting delivery dates successfully · Full statistical analysis ›

Ratings of meeting delivery dates were lowest on canceled engagements and less favorable among individual contributors than portfolio directors. The mean rating was 4.8 and the median 5.0 (n = 798 respondents). Canceled engagements rated lowest, at 3.4 (n = 304 respondents), and individual contributors rated lower than portfolio directors.

Ratings of delivery against dates were associated with all four study measures, each association holding under the applicable robustness checks. Engagements rated as meeting their dates had a lower cancellation rate (r = −0.60, 95% CI −0.70 to −0.49, n = 150 engagements). They also recorded higher customer satisfaction among active engagements (r = 0.59, 95% CI 0.44 to 0.71, n = 94 engagements). The largest observed absolute coefficient was for the engagement indicator mean (r = 0.78, 95% CI 0.71 to 0.84, n = 150 engagements). Respondents who rated delivery higher reported lower intent to leave (r = −0.61, n = 751 respondents). Distressed ratings came from 234 of the 798 answers (29.32%), and the engagement mean was distressed on 30 of the 150 engagements.

The written responses that addressed delivery dates numbered 57, from 54 respondents on 42 engagements, and 42 of the 57 were negative. In 14 responses, approvals arrived after the deadline, and the respondent decided without them or let the date slip.

“I just made the call myself, the change approval group meets barely at all and the decision always lands after the deadline its needed for, so I tweak the account data to match what the workflow expects and push it through myself before anyone signs off.” -Individual Contributor, CXM, Client servicing and communications

“In June we had a release window on a Friday evening, and the change record needed an approver from their platform team, who had gone on leave without delegating, so the deploy sat until Monday even though everything else was ready.” -Engagement Manager, ITO, Technology engineering and operations

Another 14 written responses described meeting deadlines by bypassing a process or approval, and seven described reports that concealed delays from the financial institution. These are respondent accounts, not independently inspected delivery or reporting records.

“…change process is too slow for how fast we work, so I catch it myself and keep report on time” -Individual Contributor, CXM, Regulatory, audit, tax and legal

“The delivery methodology gaps got hidden by scoping every missed milestone as a change request, which reset the baseline schedule each time and kept the reported on-time delivery rate looking healthy.” -Individual Contributor, ITO, Technology engineering and operations

“And do note it happen most weeks, i tell my lead about the same issue every time, nothing change so i just fix it and the payroll goes out on time” -Individual Contributor, BPO, Corporate and back-office functions

“The vendor reported fleet-scaling milestones as met by counting agents deployed to a test environment rather than production, so the status report showed steady progress for months while zero tickets were actually being routed live.” -Individual Contributor, CXM, Technology engineering and operations

2.2 Budget Pressure From Effort the Plan Did Not Price

Indicator C050 · The engagement is within budget · Full statistical analysis ›

More favorable assessments of delivery within budget were associated with all four core study measures. The mean rating was 4.9, and the median was 5.0 (n = 554 respondents), and 242 respondents answered: “I do not know.” Delivery within budget was associated with all four study measures, and none of the applicable robustness checks overturned an association. Engagements rated within budget had a lower cancellation rate (r = −0.54, 95% CI −0.64 to −0.42, n = 150 engagements). Their customer satisfaction among active engagements was higher (r = 0.62, 95% CI 0.48 to 0.73, n = 94 engagements). The largest observed absolute coefficient was for the engagement indicator mean (r = 0.76, 95% CI 0.69 to 0.82, n = 150 engagements). Distressed ratings came from 151 of the 554 answers (27.26%), and the engagement mean was distressed on 31 of the 150 engagements.

Seven written responses addressed the budget, each negative. Six described effort or cost that the engagement absorbed outside its plan or fee, in the form of rework, staff added under a fixed fee, repeated runs of the AI tools, and manual checks.

“In June we were asked to add two more allocation reviewers and were told the fee is fixed on the numbers agreed last year, which nobody has revisited since agents took half the run. So we absorbed it.” -Engagement Manager, BPO, Trade, settlement and reconciliation

“on my last project the bot was a script, you knew the cost. this one burns through budget re-runing the same patch check four times, nobody warned us about that part” -Individual Contributor, ITO, Client servicing and communications

“We check output every run now, it move on its own sometimes, feels like extra work nobody planned for but we do it anyway.” -Individual Contributor, BPO, Corporate and back-office functions

“Scope-creep tickets tied to mandate changes were logged under a generic 'change request' bucket, which kept the reported schedule variance small while the real rework hours piled up underneath it.” -Individual Contributor, ITO, Payments, cards and disputes

2.3 Engagements Not Achieving Stated Key Performance Indicators

Indicator C051 · The engagement is not achieving stated KPIs · Full statistical analysis ›

Individual contributors assessed progress toward the engagement's stated key performance indicators less favorably than portfolio directors. Converted to the favorable direction, on which a higher figure marks an engagement closer to its targets, the mean rating was 4.8 (n = 782 respondents). Individual contributors rated the engagement furthest from its targets, at 4.7 (n = 608 respondents), and portfolio directors closest, at 5.8 (n = 24 respondents).

Progress toward the key performance indicators was associated with all four study measures, and the associations held under every one of the applicable robustness checks. Engagements rated further from their targets had a higher cancellation rate (r = −0.59, 95% CI −0.69 to −0.48, n = 150 engagements). Engagements rated closer to their targets recorded higher customer satisfaction among active engagements (r = 0.62, 95% CI 0.48 to 0.73, n = 94 engagements). They also had a higher engagement indicator mean (r = 0.79, 95% CI 0.72 to 0.84, n = 150 engagements). Distressed ratings came from 241 of the 782 answers (30.82%), and the engagement mean was distressed on 32 of the 150 engagements.

The written responses that addressed the indicator numbered 14, from 13 respondents on 12 engagements, and 12 of the 14 were negative. Respondents described key performance indicators that were missed and service levels that declined once agents took on the tasks. Four responses, given to the question on how the vendor altered its reports, described missed targets presented to the financial institution under other headings. Part 5 examines the alteration of performance reporting.

“Missed KPI targets get reclassified as data-quality exceptions rather than performance misses, which keeps the official miss rate lower than what the team actually experiences month to month.” -Individual Contributor, CXM, Client servicing and communications

“SLAs collapsed once agents touched the work” -Engagement Manager, ITO, Client servicing and communications

“KPIs are missed almost every cycle and nobody above me seems alarmed enough” -Engagement Manager, CXM, Corporate and back-office functions

“SLA and KPI targets collapsed the moment agent output stopped matching human pace” -Individual Contributor, BPO, Trade, settlement and reconciliation

2.4 Delivery Team Knowledge of Financial Institution Key Performance Indicators

Indicator C047 · I know the KPIs the customer uses to measure success · Full statistical analysis ›

Respondents rated the statement that they knew the key performance indicators the financial institution used to measure success, with a mean rating of 4.8 and a median of 5.0 (n = 787 respondents). Canceled engagements rated lowest, and individual contributors rated lower than portfolio directors.

Self-rated knowledge of performance targets was associated with lower cancellation (r = −0.64, 95% CI −0.72 to −0.53, n = 150 engagements), higher active-engagement customer satisfaction (r = 0.65, 95% CI 0.51 to 0.75, n = 94), and a higher engagement indicator mean (r = 0.81, 95% CI 0.75 to 0.86, n = 150). The applicable sensitivity checks did not overturn these associations. The coefficient for the indicator mean was numerically largest among the four delivery indicators, without establishing an independent contribution or a statistically confirmed ranking. Distressed ratings came from 226 of the 787 answers (28.72%), and the engagement mean was distressed on 30 of the 150 engagements.

The single written response on the indicator described expectations the financial institution had not stated to the delivery team.

3 Measured Return in Production

The recorded-return field applies to the 30 Production engagements. It is not populated for the other 120 engagements. A full-sample mean would mix observed values with structural absence. The financial definition and source limitations are stated in the Introduction, section 5.5 of “Study Design, Evidentiary Basis, and Definitions.”

3.1 Measured Return Confined to Active Production Engagements

Analysis · Reported return on the engagement, ROI % · Full statistical analysis ›

3.1 Positive Measured Return Confined to Active Production Engagements

All 30 Production engagements had a return value recorded. Seventeen had positive values, averaging 24.71% and ranging from 6.00% to 47.00%. The other 13 were recorded at zero: 10 canceled engagements and three active engagements. All positive-return engagements were active. These values describe the study record, not an independent financial verification. The recorded zeros do not establish whether the respondent lacked the information, the engagement had achieved no return, or a return had not yet been realized.

Within the 17 engagements that reported a return, a higher return was associated with higher customer satisfaction (r = 0.71, 95% CI 0.35 to 0.89, p = 0.001, n = 17 engagements). A higher return was also associated with lower intent to leave (r = −0.71, 95% CI −0.89 to −0.34, p = 0.002, n = 17 engagements). The association with the engagement indicator mean was positive and inconclusive (r = 0.47, 95% CI −0.01 to 0.78, p = 0.056, n = 17 engagements), a test with power of 0.48 at this base. Every result rests on 17 engagements, and the intervals are wide.

The return is provided by the vendor delivery team. The study had no method to validate the provided figures or the level of first-hand knowledge by the respondents. The study holds no record from the financial institution against which to confirm it.

Two engagement managers, writing on the vendor's reporting, described return calculations that overstated the return. Individual contributors described efficiency figures computed without the hours staff spent checking or completing the agents' work.

“The vendor's ROI tracker recalculated the baseline savings assumption each quarter to match whatever the actual result was, so the reported ROI never fell short even as the underlying business case kept eroding.” -Engagement Manager, KPO, Corporate and back-office functions

“The vendor's monthly value report counted every hour the agent spent on forecasting as fully productive, with no line for the hours controllers then spent reconciling those forecasts against the ledger, which made the ROI case look real when it was not.” -Engagement Manager, KPO, Corporate and back-office functions

“The vendor's efficiency report calculated hours saved against a fully manual baseline without netting out the additional hours analysts spent verifying agent-flagged outputs, inflating the apparent productivity gain shown to the desk.” -Individual Contributor, KPO, Corporate and back-office functions

“The vendor reported time saved based on tasks routed to the agent, without considering the extra time staff spent manually completing the same tasks after the agent hit an API wall, which overstated the actual efficiency gain.” -Individual Contributor, CXM, Corporate and back-office functions

The vendor's client-facing performance dashboard rolls up billing accuracy as a single blended percentage across all fee types, which lets a persistent error rate on complex tiered fee calculations get diluted by the much larger volume of simple flat fees the agent handles cleanly, so the dashboard never isolates the tiered-fee problem that audit would actually care about.”

PORTFOLIO DIRECTOR · BPO, Payments, cards and disputes · “

We base the dashboards on real data, but AI still creates statistics that are not correct. We try to fix these before the customer can see but the dashboards are real time. If they look at the dashboard during their work hours, I don't know if what they see is good.”

INDIVIDUAL CONTRIBUTOR

KPO, Regulatory, audit, tax and legal