AX Field Notes

All casesNo. 053

UK DWP Measures Copilot Against Staff Without It

A comparison group put the daily saving at 19 minutes, largest in search and email

Case no.
053
Primary source
Government evaluation report
Source published
2026-01-29

1. Why this case

The Department for Work and Pensions (DWP) is a UK government department. From October 2024 to March 2025 it gave 3,549 of its central office (corporate) staff a paid Microsoft 365 Copilot licence, the version that sits inside Outlook, Teams, Word, and the other Microsoft 365 applications and can reach the department’s own documents and email. On 29 January 2026 it published an evaluation of that trial, written by two DWP economists and a DWP social researcher.

Earlier UK government evaluations of Copilot, as DWP describes them, rested on what users reported about themselves. This one also relies on self-reports, but it adds something those lacked: a survey of 2,535 staff who did not have Copilot, sampled to match the licence holders by business area and grade, and a regression that compares the two groups. The report calls this “a methodological advancement” over the government research published before it.

The headline is an estimated 19 minutes saved per day, averaged across eight routine tasks. The more useful parts are underneath it: how that number moved as controls were added, which tasks gained most and least, what staff did with the time, and where Copilot did not help, notably in Excel and in work that had to be checked again by hand.


2. Problem

Before the trial, in summer 2024, DWP set out a theory of change for Copilot. It concluded that Copilot “has the potential to deliver efficiency gains by reducing time spent on routine tasks and enabling staff to focus on more complex work,” and listed the routine work it expected to streamline: “generating meeting notes, summaries, emails, reports, and coding outputs.” The same exercise named the risks: the learning curve, “the risk of over‑reliance on Copilot,” and “inappropriate decision‑making if outputs are not subject to sufficient human oversight.”

The interviews describe where that routine load sat for central office staff. Summarising “was especially valued by those needing to catch up on communications after periods of absence or when faced with information overload.” Staff in secretariat or administrative roles carried “the burden of manual notetaking” in meetings, which limited how actively they could take part.

The department also had a decision problem. The publication page says the evaluation was meant to “ensure future decisions on investment, training, and policy surrounding AI adoption are evidence-based.” Earlier government evaluations of Copilot, by the Department for Business and Trade and the Government Digital Service, had reported time savings, but in DWP’s description both relied on self-reports without a comparison group. DWP’s evaluation instead compared users with a large group of non-users, an approach it says “enables a more rigorous assessment of Copilot’s potential impact.”

Two boundaries applied. The trial covered central office staff only; “frontline operational colleagues (such as those in Jobcentres) did not receive any access to Microsoft Copilot during this trial.” And use was governed by a departmental policy framework covering “security, data protection, and acceptable use.”

Confirmed facts: theory of change drafted summer 2024; expected gains in routine tasks (meeting notes, summaries, emails, reports, coding outputs); named risks include over-reliance and decisions without sufficient human oversight; evaluation intended to inform investment, training, and policy decisions; scope = central office (corporate) staff, no Jobcentre frontline staff; earlier DBT and GDS evaluations described by DWP as self-reported with no comparison group.


3. Solution

3.1 Who got a licence

DWP allocated 3,549 licences “through a mix of volunteers and peer nominations,” on what the report calls “a first-come, first-served basis (often through managerial discretion or peer nomination).” The licences were “divided into 12 licence groups aligned with specific business areas (for example Digital, Policy, Finance) to ensure broad coverage of the department’s functions.”

The licensed version “is integrated into Microsoft 365 applications and provides enhanced features, including access to organisational data, advanced security, and compliance capabilities.” A free version, which “lacks integration with DWP data,” was made available to all DWP staff in April 2025, after the trial. The evaluation covers only the licensed version.

3.2 How staff learned it

Training existed: online resources, demonstration sessions, and support materials. In practice, most people learned on their own. Of 1,713 licence holders who answered the survey, 89% learned through self-directed exploration, 77% used DWP-provided resources, 61% learned from peers, and 56% used external internet sources. Most interviewees “did not remember attending any formal sessions.”

Two internal channels carried much of the load. One user described the onboarding this way: “as soon as we got the licence, you immediately get access to that Microsoft Copilot Teams channel anyway, where there is loads of information.” The report names the Copilot Teams channel and a SharePoint site as particularly useful. Adoption spread through colleagues: “Early adopters often became internal champions, informally discussing Copilot’s features with colleagues and performing live demonstrations.” Users asked for shorter, role-specific sessions and a “cheat sheet” that included “a list of prompts tailored to DWP.”

3.3 What Copilot took on, and what staff kept

Use was frequent: 65% of surveyed licence holders used Copilot daily and 30% weekly. The most common uses were creating meeting notes, searching for information, and summarising documents, followed by drafting messages and, for some, data analysis.

The interviews show how the work was split in each of these.

  • Meeting notes. Copilot produced the minutes; staff checked and adjusted them. One user said: “I barely have to update the notes. They pretty much come out fully accurate. It means you can participate fully in the meeting without having to worry too much.” In the survey, 85% rated the accuracy of Copilot’s Teams meeting summaries as good or very good.
  • Drafting. Staff supplied the substance and Copilot wrote the text. “I pull out the key information that I need and get Copilot to create the Comms.” Users valued it for changing tone in sensitive or complex messages.
  • Search. Copilot found documents “saved in SharePoint libraries or personal OneDrive accounts,” searched email, and defined acronyms. Users were more cautious with internet results: “it cannot be used as an official source.”
  • Coding. A small number of data analysts used it to create or refine code faster.

Across all of these, the human step stayed in place. “Even when Copilot performed well, users consistently reviewed and edited its outputs before using them.” The report presents this as professional practice rather than distrust: “most users felt responsible for the final product.” One user put it as: “I always know to cross check things anyway and never use it for the final document.”

Staff also left some work out. Several participants were “cautious about inputting internal or confidential DWP data into Copilot, following organisational guidance.” Users “noted functional limitations, especially in Excel and, to a lesser extent, PowerPoint,” where Copilot “was seen to struggle with data-heavy tasks, including interpreting complex datasets, generating charts, and managing intricate formatting.” Search sometimes returned irrelevant results “based on keyword matches rather than the full content of the prompt.”

3.4 How the trial was measured

The evaluation combined two surveys, an econometric comparison, and 19 interviews.

  • Treatment group: all 3,549 licence holders were invited; 1,716 responded (48%), between 14 and 28 February 2025.
  • Comparison group: 9,300 staff without a licence were invited, using stratified random sampling so that “licence group and grade distribution closely matched those of the treatment group”; 2,535 responded (27%), between 4 and 14 March 2025.
  • Interviews: 19 licence holders, chosen by quota across licence groups, interviewed for about an hour each between 5 and 14 March 2025.

An analysis plan with eleven key tests was fixed before data collection, with a Bonferroni correction for multiple testing. The main model was a Seemingly Unrelated Regression (SUR), which estimates the three outcomes (task efficiency, job satisfaction, and quality of work) together. Controls were added in nine steps, starting with none and adding, in turn, Director General group, grade, gender, age, job category, health condition, interest in AI, and current experience with AI tools. The ninth specification, with all controls, was named in advance as the headline.

Task efficiency was defined as “a calculated average of the time spent per day across 8 routine tasks,” with each answer recorded as “the lower bound of each time category from a 6-point ordinal scale (for example 15 to 30 minutes was recorded as 15 minutes).”

Confirmed facts: 3,549 licences, 12 licence groups, volunteers and peer nominations, first-come first-served; licensed version reaches organisational data; free version to all staff April 2025; 65% daily and 30% weekly use (base 1,713); learning channels 89% / 77% / 61% / 56%; 85% rated meeting-note accuracy good or very good; treatment 1,716 of 3,549 responded, comparison 2,535 of 9,300; 19 interviews; pre-specified plan with eleven tests; SUR with nine control specifications.


4. Impact

All figures below are from the DWP report published 29 January 2026, based on surveys run in February and March 2025. Model estimates use the pre-specified ninth specification unless noted.

Time. Copilot users were estimated to save “an average of 19 minutes per day across eight routine tasks” compared with similar non-users (95% confidence interval 17 to 22 minutes). The report labels the coefficient a lower bound in minutes. Without controls, the estimate was 10 minutes. It stayed between 10 and 12 minutes as demographic and job controls were added, then rose to 19 minutes when interest in AI was added. The report’s explanation: “If we do not account for these characteristics, the estimated effect of Copilot looks smaller because AI-keen users may spend more time on tasks in the absence of Microsoft Copilot.”

The task-level estimates (minutes per day, with 95% confidence intervals) were:

TaskEstimate95% CI
Searching for existing information or research2621–30
Writing an email2520–30
Summarising information or research2420–28
Producing or editing written materials2015–24
Scheduling meetings1916–23
Checking own writing for grammar, tone, or spelling1713–21
Data analysis or coding1512–19
Transcribing or summarising meetings95–13

All were statistically significant at the 1% level.

Satisfaction and quality. On 7-point scales, job satisfaction rose by an estimated 0.56 points (95% CI 0.47 to 0.65) and perceived quality of work by 0.49 points (0.42 to 0.55), both described as medium effects. The quality estimate barely moved across specifications; the satisfaction estimate fell from 0.70 unadjusted to 0.56 with all controls.

What licence holders reported. About 90% of surveyed licence holders said Copilot saved them time: 30% put it at more than an hour a day, 33% at 30 to 60 minutes, and 27% at under 30 minutes. 95% were very or somewhat satisfied with its overall performance, 65% agreed it made them feel more fulfilled at work, and 73% agreed their work quality had improved. The report notes that the econometric estimates “are generally more conservative” than self-reports.

Where the time went. Of 1,541 respondents to the question, 89% used the time saved on other work tasks, and around 60% used it to improve the quality of their work or to plan ahead. The interviews describe a shift, not a smaller workload: “Rather than reducing their overall workload, Copilot allowed them to complete administrative tasks more quickly, freeing up capacity to focus on other tasks, such as project delivery, strategic planning, or supporting senior leaders.” Some also relied less on colleagues: “it’s probably saved me time, but also saved them time in a way.”

Where it did not pay off. “A few users raised concerns about double-handling and verification overhead.” For them, “in some cases, this additional step negated the time savings.” One said: “I just felt if I had done it manually from the beginning, it would have been better.”

The report publishes no headcount, staffing, cost, or licence-price figures, and no decision on licensing after the trial.


5. Insight

Self-reports and the comparison estimate are different measures. Thirty percent of licence holders said Copilot saved them more than an hour a day. The modelled figure against matched non-users is 19 minutes per day, averaged across eight routine tasks, with each answer recorded at the bottom of its time band. The report says its econometric estimates “are generally more conservative” than self-reports. For a department deciding whether to pay for licences, the comparison estimate is the one built to be defended.

Controlling for enthusiasm raised the estimate. The usual worry with volunteer trials is that keen people report better results, and the report accepts that this may lead to overestimation. But in the time model, adding interest in AI moved the estimate from 11 to 19 minutes. DWP’s reading is that AI-keen staff would spend more time on these tasks anyway, so leaving that out hides part of the effect. Licence holders differed from non-users far more on AI interest and experience (Cramér’s V 0.481 and 0.338) than on grade or business area (0.107 and 0.121), so this control carries much of the weight.

Gains followed text work. The largest estimates were in search, email, and summarising. Transcribing or summarising meetings had the smallest estimate, 9 minutes, even though meeting notes were among the most common uses and 85% rated their accuracy well. The report does not explain the gap. It is a reminder that the most visible use is not necessarily where the measured time goes.

The time stayed in the job. The report describes capacity moving to project delivery, planning, and support for senior leaders, not fewer hours or fewer people. It publishes nothing on staffing or cost. What DWP measured is freed time inside existing roles.

Checking is part of the work. Staff reviewed and edited outputs as a matter of course, kept confidential data out by their own judgement under departmental guidance, and found Copilot weak in Excel and data-heavy work. For a few users, checking took back the time saved.

Adoption ran through people, not courses. Licences came through volunteering, managers, and peer nominations. Most staff learned by trying it, from a Teams channel and from colleagues’ demonstrations, and asked for role-specific sessions and DWP prompt lists that did not yet exist.

Limits. The evaluation was carried out by DWP’s own analysts. Licences were not randomly assigned, there was no pre-trial baseline, and all outcomes, including time, were self-reported. Response rates were 48% among licence holders and 27% in the comparison group. The trial covered central office staff only, used the Copilot version of 2024–25, and its interviews (19 people) are illustrative rather than representative.

One-line takeaway: DWP’s trial shows that a workplace AI assistant measured against matched non-users gives a more conservative daily estimate than users’ own reports, concentrated in search, email, and summarising, with the freed time moving to other work and the checking still done by staff.


6. Implementation hypotheses (not in the source)

6.1 Licence groups as the unit for allocation, support, and sampling

Basis: The 3,549 licences were “divided into 12 licence groups aligned with specific business areas”; the comparison sample was matched on “licence group and grade”; interviews used licence group as the quota stratum; one user got the Copilot Teams channel “as soon as we got the licence.”

The licence groups were probably directory groups, one per business area, each holding a share of the 3,549 licences that filled as managers and peers nominated people. Adding someone to a group would assign the licence and, in the same step, add them to the Copilot Teams channel and the SharePoint support site, which would explain why a new licence holder found the channel already there. The same group label then likely served the evaluation: analysts could map every non-licensed employee to the business area of the matching licence group, combine that with grade from HR records, and draw the 9,300 comparison invitations to the same mix. The three licence holders excluded because they had no Copilot on their work laptops suggest the group list and the devices did not fully line up, so the census was probably drawn from the licence list rather than from usage logs.

6.2 Copilot’s reach into DWP data, and the line staff drew themselves

Basis: The licensed version includes “access to organisational data”; users found documents in “SharePoint libraries or personal OneDrive accounts”; search sometimes matched on keywords “rather than the full content of the prompt”; staff were “cautious about inputting internal or confidential DWP data into Copilot, following organisational guidance.”

Copilot most likely answered from what each user could already open: their mailbox, their OneDrive, Teams meetings they attended, and the SharePoint libraries their account had access to. That would make search quality depend on how existing libraries were named and permissioned, not on a curated DWP knowledge base, and it fits the complaint that a single matching word could pull in an irrelevant document. The data boundary appears to have been held mainly at the point of input, through guidance and each user’s judgement about what to paste or ask, rather than through a technical block. Under that design the policy framework and the user’s own caution do the work, which is consistent with staff describing confidentiality as a reason they used Copilot less.

6.3 How the 19-minute figure was assembled

Basis: Task efficiency is “a calculated average of the time spent per day across 8 routine tasks,” with minutes “expressed as the lower bound of each time category from a 6-point ordinal scale”; Table 4.4 gives eight task estimates from 9 to 26 minutes; the headline is 19.

Both surveys likely asked respondents to place each of the eight tasks into the same six time bands. Each answer would be converted to the band’s lower bound, the eight values averaged for each person, and that average regressed on licence status with the controls. When the same respondents and controls are used, a coefficient on an average equals the average of the task coefficients, and the eight task estimates (17, 15, 20, 25, 19, 26, 24, 9) average about 19.4. The headline is therefore best read as the gap on a typical routine task, not as the sum of eight task savings or as a measured share of the working day. Recording each answer at the bottom of its band also means the minutes are floors on what respondents reported, which is why the report labels the coefficient a lower bound.


7. Sources

  1. Department for Work and Pensions — An Evaluation of DWP’s Microsoft 365 Copilot Trial (Francesco Arzilli, Callum Seán Lynch, Lucy Page)
    https://www.gov.uk/government/publications/an-evaluation-of-dwps-microsoft-copilot-365-trial/an-evaluation-of-dwps-microsoft-365-copilot-trial
    2026-01-29 (accessed 2026-11-17)
  2. Department for Work and Pensions — An Evaluation of DWP’s Microsoft Copilot 365 Trial (publication page)
    https://www.gov.uk/government/publications/an-evaluation-of-dwps-microsoft-copilot-365-trial
    2026-01-29 (accessed 2026-11-17)