What problem does it solve?
Large organizations run thousands of engineers, and a big share of their time goes to work that is necessary but not differentiating: boilerplate, glue code, tests, reading code someone else wrote, and first pass review. Banks and insurers add a long tail of internal frameworks and legacy services that new joiners have to learn before they are productive.
Coding assistants take part of that load. The question for an engineering leader is no longer whether developers will use them, but how to capture the gain safely: keeping proprietary code out of external training, stopping insecure or unlicensed code from reaching production, and measuring real delivery rather than lines of code accepted.
How does it work?
- Inline completion and chat in the IDE. The assistant suggests code as the developer types and answers questions about the code base, using the open files and repository as context.
- Tests and explanations. Developers ask it to draft unit tests, explain a module or propose a fix for a failing build.
- Review assistance. On a pull request it summarizes the change and flags likely defects, which the human reviewer accepts or rejects.
- Normal controls apply. Generated code goes through peer review, static analysis, secret and licence scanning, tests and change approval, exactly like human code.
- Agentic tasks, carefully. Newer tools can plan and apply multi file changes or run commands. These should run in sandboxes with limited permissions, never directly against production.
- Audience
- Employee facing
- Autonomy
- Copilot
- Adoption
- Mainstream
- Channels
- Internal tools
What is it worth?
Benchmarks are computed from the public deployments below: one data point per organization per KPI, with who made each claim.
| KPI | Median | Reported range | Data points | Claimed by |
|---|---|---|---|---|
| Productivity gain | 22.5% | 8.7% to 42.4% Not pooled: up to 30% | 4plus 1 up to | 2 organization, 2 vendor |
| Hours saved | Not pooled | at least 10.5 hours | 1 | 1 vendor |
| Users served | Not pooled | at least 1000 | 1 | 1 vendor |
Value drivers: Employee productivity, Speed and cycle time.
Indicative value
An engineering organization with 500 developers
USD 1.5 million to USD 7.5 million
Developer capacity released per year
How this is calculated
Formula: developers * loadedCost * affectedShare * timeSaved. The low scenario uses every low input, the high scenario every high input.
| Input | Low | High | Basis |
|---|---|---|---|
| Developers with the assistant developers, developers | 500 | 500 | The reference organization. |
| Fully loaded cost per developer loadedCost, USD per developer per year | 100,000 | 150,000 | Editorial assumption. Replace with your own blended cost, including contractors. |
| Share of developer time spent on tasks the assistant helps with affectedShare, fraction of working time | 0.3 | 0.5 | Editorial assumption. Coding, tests and reading code, excluding meetings, design and incidents. |
| Time saved on those tasks timeSaved, fraction of task time | 0.1 | 0.2 | Editorial assumption, set below the Bank of America and ANZ figures on this page (Bank of America reports efficiency gains of over 20%; ANZ's controlled experiment measured about 42% less time on algorithmic Python challenges). Most CME Group developers using Gemini Code Assist report at least 10.5 hours a month, which falls inside this range. The only field randomized trial, Accenture's, measured a different thing (8.69% more pull requests), so replace this with your own control group result. |
What it leaves out: Released capacity, not cash: it only becomes value if the time goes into more delivery. It leaves out licence and review costs, the extra review effort generated code can create, and quality effects in either direction.
Who already uses it?
10 public deployments, strongest evidence first. Grades: A regulator or audit, B the organization itself, C vendor case study, D anonymous or estimate.
Bank of America
United States · Banking · 2025
Bank of America software developers use a generative AI tool that helps them write and optimize code. The bank reported the efficiency gain in its April 2025 update on how its workforce uses AI, alongside its internal assistants and contact centre tools. The bank does not name the underlying model or vendor.
- Productivity gain: at least 20%
"Coding assistance – Bank of America software developers are using a GenAI-based tool to assist with code writing and optimization, through which they have experienced efficiency gains of over 20%."
Claimed by: organization
ANZ
Australia · Banking · 2024
ANZ, which employs over 5,000 engineers, ran a six week experiment with GitHub Copilot from June to July 2023: after two weeks of preparation, over 100 participants were split at random into a control group and a Copilot group that solved the same algorithmic Python challenges. The Copilot group took markedly less time and produced code with fewer code smells and bugs, while the effect on security was inconclusive. By the time of writing about 1,000 engineers were using Copilot. The study was written by ANZ staff and published on arXiv.
- Productivity gain: 42.4%, two week A/B test on algorithmic Python challenges, self reported time
"This study shows that Copilot improves the productivity of ANZ engineers by 42.36% on an average."
Claimed by: organization
Citigroup
United States · Banking · 2024
On Citi's fourth quarter 2024 earnings call, CEO Jane Fraser said the bank had armed 30,000 developers with AI tools to write code and launched two AI platforms for 143,000 colleagues. CIO Dive reported the same rollout as generative AI coding tools. No productivity or quality figures for the coding tools are disclosed, and the vendors are not named.
No outcome disclosed.
Meta
United States · Technology and software · 2024
Meta built TestGen-LLM, a tool that uses large language models to draft and improve unit tests. During Instagram and Facebook test events, it improved 11.5% of all classes it was applied to, and 73% of its recommended test improvements were accepted by Meta software engineers for production deployment.
- Accuracy: 73%
"During Meta's Instagram and Facebook test-a-thons, it improved 11.5% of all classes to which it was applied, with 73% of its recommendations being accepted for production deployment by Meta software engineers."
Claimed by: organization
AMD
United States · Technology and software · 2026
Chip maker AMD consolidated more than 20,000 hardware and software engineers onto GitHub Enterprise and deployed GitHub Copilot, including Copilot coding agent and Copilot Autofix, across presilicon verification and security remediation. GitHub's own case study reports that a historical agent built on Copilot Autofix now resolves approximately 70% of vulnerability classes automatically, that agentic workflows lift AI generated code acceptance above 50% on some projects compared with roughly 30% for standard completion, and that Copilot's coding agent automates the overnight triage of AMD's roughly one million nightly chip design simulations.
- Automation rate: about 70%, vulnerability classes handled by the Copilot Autofix "historical agent"
"approximately 70% of vulnerability classes are now resolved automatically"
Claimed by: vendor
Cathay Pacific
Hong Kong SAR China · Travel and hospitality · 2025
Cathay, the airline group that operates Cathay Pacific, rolled out GitHub Copilot, agent mode and Copilot Autofix as it unified development and security on the GitHub platform. GitHub's own case study reports that Cathay rolled Copilot out to more than 1,000 developers in one week, who have since accepted over four million lines of Copilot generated code, that satisfaction and NPS scores among developers rose to 4.4 out of 5, and that catching vulnerabilities earlier in the workflow, with Copilot working together with GitHub Advanced Security, cut the mean time to remediate security fixes by 63%.
- Users served: at least 1000, developers onboarded to GitHub Copilot within one week of rollout
"Cathay rolled out GitHub Copilot to more than 1,000 developers in just one week, who have since accepted over four million lines of code."
Claimed by: vendor - Time to repair reduction: 63%, mean time to remediate security vulnerabilities, after Copilot and GitHub Advanced Security
"This shift to proactive security has led to a 63% decrease in the Mean Time to Remediate (MTTR) security fixes."
Claimed by: vendor
CME Group
United States · Capital markets · 2025
CME Group, which operates the Chicago Mercantile Exchange, gave its developers Gemini Code Assist. Google Cloud reports that most developers using it say they gain at least 10.5 hours a month.
- Hours saved: at least 10.5 hours, per developer per month, self reported
"CME Group, which operates the Chicago Mercantile Exchange, says most developers using Gemini Code Assist report a productivity gain of at least 10.5 hours a month."
Claimed by: vendor
Accenture
Ireland · Professional services · 2024
GitHub and Accenture ran a randomized controlled trial in which Accenture developers were randomly given GitHub Copilot or not, and measured DevOps telemetry such as pull requests and build success. The Copilot group opened more pull requests and saw many more successful builds, and surveyed developers reported less mental effort on repetitive tasks. GitHub also ran a company wide adoption analysis of installation and suggestion acceptance at Accenture.
- Productivity gain: 8.7%
"Ultimately, an increase in pull requests represents an increase in value delivered, and Accenture developers saw an 8.69% increase in pull requests."
Claimed by: vendor
Hitachi, Ltd.
Japan · Technology and software · 2024
Hitachi deployed GitHub's services internally from 2021, later extending that use to GitHub Copilot, and began a structured internal evaluation in October 2023 with about 200 developers over three to four months using the SPACE productivity framework. Hitachi also integrated Copilot with its own development framework, Justware, so developers use Copilot to turn Justware's generated skeleton code and design documents into working business logic. Hitachi has since grown Copilot usage to about 2,000 users and runs a company wide Generative AI Practitioner Community to share what teams learn.
- Productivity gain: up to 30%, coding and unit testing, internal use
"Productivity gains average 10 to 20 percent in these areas, reaching 30 percent in some cases."
Claimed by: vendor
Duolingo
United States · Technology and software · 2023
Language learning company Duolingo gave its 300 developers GitHub Copilot alongside Codespaces and a custom GitHub bot integration built on GitHub's API. GitHub's own case study reports that a senior engineering manager estimates at least a 25% increase in developer speed for engineers new to a repository, and about 10% for those already familiar with it, mostly from faster boilerplate code, alongside a drop in median code review turnaround time and a rise in pull request volume that the case study credits to the platform as a whole rather than Copilot alone.
- Productivity gain: at least 25%, self reported estimate for developers new to a specific repository or framework
"Burket estimates at least a 25% increase in developer speed, and a 10% increase for those already familiar with that same codebase, who can more quickly and easily create boilerplate code."
Claimed by: vendor
How do you implement it?
A model agnostic playbook: what to prepare, the order to build in, and what goes wrong.
Data you need
- Repository access rules and a list of repositories excluded from assistant context
- Secure coding standards and approved libraries the assistant should follow
- Baseline delivery metrics (cycle time, pull request throughput, change failure rate)
Systems to integrate
- IDEs and the source control platform
- CI pipeline with static analysis, secret scanning and licence scanning
- Identity provider for licence assignment and single sign on
- Model gateway or vendor tenant with zero retention terms
Complexity: Low
Rolling out a commercial assistant is technically simple. The effort is in the contract and data terms, security review, secure configuration of repositories and secrets, training, and a measurement plan that looks at delivery rather than acceptance rates.
- 1
Settle data terms first
Choose a tenant where your code is not retained or used for training, confirm where prompts are processed, and record the tool as a third party service in your risk register.
- 2
Pilot with a control group
Give the tool to a representative set of teams and compare them with similar teams without it over several weeks, on delivery metrics rather than surveys alone.
- 3
Strengthen the pipeline
Make secret scanning, dependency and licence checks and static analysis mandatory gates before merge, since more code will arrive faster.
- 4
Train reviewers as well as authors
Teach engineers to treat suggestions as untrusted input, to check generated tests actually test something, and to reject code they do not understand.
- 5
Scale and keep measuring
Roll out by team, track adoption and delivery metrics per cohort, and revisit settings when new agentic features arrive.
Guardrails
- Zero retention and no training on the organization's code, confirmed contractually
- Generated code passes the same review, testing and change approval as human code
- Mandatory secret, dependency, licence and static analysis scanning before merge
- Agentic features run in sandboxes without production credentials
- Sensitive repositories excluded from assistant context where required
KPIs to instrument
- Pull request throughput and lead time per team, compared with a control group
- Change failure rate and escaped defects
- Security findings per thousand lines in generated versus human code
- Weekly active users among licensed developers
- Developer satisfaction, surveyed quarterly
Human in the loop
A developer accepts or rejects every suggestion and remains the author of record. A second engineer reviews every change before merge, and release managers approve production changes as before. Agent generated multi file changes are reviewed like a new colleague's first pull request.
Common failure modes
- Measuring the wrong thing
- Acceptance rates and lines generated rise while delivery does not. Measure throughput, lead time and quality against a control group.
- Faster insecure code
- Suggestions reproduce insecure patterns or hard coded secrets. Scanning gates and reviewer training catch them; the tool alone does not.
- Source code leakage
- Engineers paste proprietary code into public chat tools when the approved tool is weak. Provide a good approved tool and block the alternatives.
- Agents with too much reach
- An agent with shell or database access acts outside its task. Limit permissions and never give it production credentials.
What are the risks and rules?
EU AI Act
Minimal risk
A coding assistant used by developers is not a prohibited practice under Article 5 and is not listed in Annex III. Developers know they are working with an AI tool, so the Article 50 disclosure duty has no practical effect for the deploying organization, and the marking of generated content under Article 50(2) falls on the tool's provider. What remains is AI literacy (Article 4). Using an AI system to monitor or evaluate individual developers' performance would fall under Annex III point 4(b), and the software the assistant helps build may itself fall under the Act.
Guidance
- SP 800-218, Secure Software Development Framework (SSDF) Version 1.1 (NIST, North America). Baseline secure software development practices, such as code review, testing and vulnerability response, that apply to generated code as much as to code written by hand.
- Guidelines for secure AI system development (UK National Cyber Security Centre, Europe). Guidelines for providers of AI systems in four areas (secure design, development, deployment, and operation and maintenance), relevant when you build your own tooling or agents around the assistant.
- Article 4, AI literacy (European Union, Europe). Providers and deployers must take measures on the AI literacy of staff who use AI systems (the amended wording shown on this page asks them to support it rather than ensure a sufficient level). Here that means training developers on the tool's limits.
Controls to put in place
- Third party risk assessment and register entry for the assistant vendor
- Contractual zero retention and data location terms
- Mandatory scanning gates in the CI pipeline
- Developer training on reviewing generated code
- Periodic review of agent permissions and repository exclusions
When it went wrong elsewhere
- Incident 768: ChatGPT reportedly implicated in Samsung data leak of source code and meeting notes. Samsung engineers were reported to have leaked source code and internal meeting notes in March 2023 by entering them into ChatGPT for help with their work. It shows why an approved tool with proper data terms matters.
- Incident 1152: LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze, Leading to Loss of Production Data. An agentic coding tool with access to a live production database reportedly deleted it during a code freeze, despite instructions not to change code without permission. Agentic features need sandboxes and least privilege.
Frequently asked questions
- How much faster do developers get with a coding assistant?
Published results vary widely with how they are measured. Bank of America reports efficiency gains of over 20% for its developers, Accenture's randomized trial with GitHub saw 8.69% more pull requests, and ANZ measured about 42% less time on algorithmic Python challenges. Set exercises tend to overstate everyday gains, so measure your own teams against a control group.
- Does our code train the vendor's model?
It depends on the vendor, the plan and the contract, and terms differ between them. Confirm in writing that your code is not used for training and how long prompts are retained, check where prompts are processed, and treat the tool as a material third party service where your regulator expects that.
- Is generated code a regulatory problem for a bank?
Not in itself. Rules such as DORA require ICT change management, testing and security controls, and these apply whoever or whatever wrote the code. The risk is the pipeline receiving more code than it can review, so strengthen scanning and review before scaling.
How to cite this page
Blits.ai AI Use Case Library, "AI coding assistant for software developers", last verified 27 September 2026, https://www.blits.ai/ai-use-cases/developer-coding-assistant. Licensed under CC BY 4.0. Method: how we verify use cases.
Changelog
- 1 October 2026: Added deployments: Hitachi.
- 29 September 2026: Added deployments: Duolingo, AMD, Cathay Pacific.
- 27 September 2026: First published