Why This Question Stalls AI Roadmaps in 2026
Yes, you can build AI with existing software team members, up to a clear line. Generalist engineers ship the application layer well: LLM features, prompt UX, internal tools and fixed-path agent workflows on model APIs. They stall at the production layer, in retrieval quality, evals, monitoring, security, governance and inference cost control.
This guide from FutureProofing, which builds AI products and internal software inside established companies' systems, maps where that line falls, what learning on the job really costs, and how to close the gap.
The question stalls roadmaps because it usually gets asked after a demo works. The demo proves the application layer. It says nothing about the production layer, which is where abandoned AI projects die.
- Adoption is not the bottleneck. Google's DORA research puts AI adoption among software development professionals at 90%, from a survey of nearly 5,000 technology professionals (Google DORA, 2025).
- Usage sits in familiar work, not production. Developers who use AI mainly apply it to generating code in familiar areas (67%), and only 20% use it for deploying, operating or troubleshooting production systems, according to the Stack Overflow Developer Survey 2026 results. Writing code with AI and engineering AI that runs in production are different skills.
- The skills gap is what kills projects. Pluralsight's 2025 AI Skills Report, a survey of 1,200 executives and IT professionals in the US and UK, found that 65% of organisations have had to abandon AI projects due to a lack of AI skills (Pluralsight, 2025). That is a share of organisations, not a share of projects.
- Teams overrate their own readiness. The same Pluralsight report found that 92% of respondents were confident they had the skills to integrate AI tools into their role, while 79% admitted pretending to know more about AI than they do. That paradox is why "can my team do this?" so often gets a confident yes, followed by a stalled pilot.
- Analysts expect heavy attrition. Gartner predicted in 2025 that more than 40% of agentic AI projects will be canceled by the end of 2027 because of escalating costs, unclear business value or inadequate risk controls (Gartner, 2025).
What Generalist Engineers Can Ship Today
The question "can software engineers build AI?" has a clear answer at the application layer: yes, with modest ramp-up. Foundation models moved most AI product work from training models to calling them, which turned much of it into ordinary software engineering.
In "The Rise of the AI Engineer," swyx argued that AI tasks that once took "5 years and a research team" now "just require API docs and a spare afternoon" (Latent Space, 2023). The same essay draws the line this page follows: ML engineers own foundation-model concerns, while AI engineers own "product-specific data and evals."
Simple patterns favour generalists. Reviewing agent builds with customers, Anthropic reported that "the most successful implementations weren't using complex frameworks or specialized libraries" and advised "finding the simplest solution possible, and only increasing complexity when needed" (Anthropic, Building effective agents).
What a generalist team ships well:
- LLM features inside existing products: summarising, classifying, extracting fields from documents, drafting replies, and question answering over a small, clean corpus.
- Prompt UX: input design, streaming output, citations in the interface, human review steps, and fallbacks when the model refuses or times out.
- Internal tools: support copilots, sales-research assistants, report generators and back-office automations that call existing APIs.
- Structured output and tool calls: model output turned into JSON that existing services validate, and model calls to well-defined internal endpoints.
- Faster delivery of their own code: a Microsoft study of its early-2026 rollout of Claude Code and GitHub Copilot CLI found that adopters merged roughly 24% more pull requests than they would have otherwise (Murphy-Hill, Butler and Savelieva, Microsoft, 2026).
Tool fluency is not the same as AI engineering depth. In an IBM survey of more than 1,000 US enterprise AI developers, fielded by Morning Consult, less than one quarter (24%) of application developers ranked themselves as experts in generative AI (IBM, 2025).
LLM Features and Agentic Workflows
Generalist teams are strongest at workflows, where code decides the path, and weakest at open-ended agents, where the model decides. Anthropic defines workflows as "systems where LLMs and tools are orchestrated through predefined code paths" and agents as "systems where LLMs dynamically direct their own processes and tool usage," and notes that "agentic systems often trade latency and cost for better task performance" (Anthropic, 2024).
- Workflows a generalist team can own: prompt chaining (draft, check, rewrite), routing requests to different prompts or models, parallel calls with a vote or merge step, and orchestration over a fixed set of internal tools. Conventional engineering habits can test all of them.
- Where agents get harder: open-ended tool use against production systems, long-running multi-step tasks and anything with write access. For these, Anthropic recommends "extensive testing in sandboxed environments, along with the appropriate guardrails."
- Agents are still early for most teams: in the Stack Overflow Developer Survey 2025, 14.1% of all respondents used AI agents daily at work, and the survey concluded that "AI agents are not yet mainstream."
A useful test: if the feature can be written as a fixed sequence of steps with a human checking the output, a generalist team can usually build it. If it acts on its own, touches customer data or writes to production systems, it needs the production disciplines covered next.
Where Generalist Teams Stall
Generalist teams stall in the gap between a demo that works and a system that keeps working. That gap is made of disciplines conventional software teams rarely practise: measuring non-deterministic output, tuning retrieval, and governing data and spend.
This is where the generalist vs specialist AI engineers question gets concrete. The difference is not model training. It is production discipline. The practitioners behind "What We've Learned From A Year of Building with LLMs," including Eugene Yan, Hamel Husain, Jason Liu and Shreya Shankar, put it directly: "creating products and systems that are effective, beyond a demo, remains deceptively difficult" (Applied LLMs, 2024).
The warning signs already show up in developer data:
- "Almost right" output. In the Stack Overflow Developer Survey 2025, the top frustration (66%) was "AI solutions that are almost right, but not quite," and 45.2% said debugging AI-generated code is more time-consuming.
- Teams feel faster than they measure. METR's randomized trial of 16 experienced open-source developers on 246 real tasks found that with early-2025 AI tools they took 19% longer, while believing afterwards that AI had sped them up by 20% (METR, 2025). METR now treats that result as a snapshot of an earlier period and says current tools likely help more, though its newer data gives only a weak signal (METR, 2026). The durable lesson is the perception gap.
- AI amplifies existing practice. Google's 2024 DORA report found that as AI adoption increased, it was accompanied by an estimated 7.2% reduction in delivery stability and an estimated 1.5% decrease in delivery throughput (Google Cloud, DORA 2024). The 2025 report frames AI as "a mirror and a multiplier" that boosts cohesive organisations and exposes the weaknesses of fragmented ones (Google DORA, 2025).
Retrieval, Evals, and Production Reliability
Retrieval quality, evaluation and quality monitoring are engineering disciplines with measurable levers, not libraries you install. They are where a working demo most often fails to become a dependable system.
Retrieval is measurable engineering.
- The Applied LLMs authors write that "RAG is only as good as the retrieved documents' relevance, density, and detail," and that "in most cases, a hybrid will work best: keyword matching for the obvious matches, and embeddings for synonyms" (Applied LLMs, 2024).
- Anthropic's Contextual Retrieval work shows how far the levers move results: combining contextual embeddings, contextual BM25 and reranking cut the top-20-chunk retrieval failure rate by 67%, from 5.7% to 1.9% (Anthropic, 2024). A team that ships plain vector search and never measures its failure rate does not know it has one.
- OWASP lists "Vector and Embedding Weaknesses" as LLM08 in its Top 10 for LLM Applications 2025, so retrieval is a security surface too.
Evals are the biggest gap between generalist and specialist practice.
- Hamel Husain writes that "unsuccessful products almost always share a common root cause: a failure to create robust evaluation systems" (Hamel Husain, 2024). He describes three levels: unit tests on every change, model and human evaluation with logged traces, and A/B testing.
- Gergely Orosz and Hamel Husain explain why this feels foreign to conventional engineers: test-driven development "works because for a given input, there is a single, deterministic, knowable, correct output to assert against," which LLMs do not provide (The Pragmatic Engineer, 2025). They call error analysis "the single highest-ROI activity in AI development" and warn against "vibes-based development."
- In LangChain's 2025 survey, quality was the top barrier to production (32%), yet only 52.4% of organisations ran offline evaluations on test sets while 89% had some observability (LangChain, 2025). Many teams can see what their agents do but cannot say whether it is right.
Production reliability means monitoring quality, not just uptime. Model providers update models, prompts drift as features change, and inputs shift as real users arrive. Without a regression eval suite and trace review, a team learns about degradation from customer complaints. Environment separation matters too: in July 2025 a Replit AI agent deleted the live production database of SaaStr founder Jason Lemkin's app during a declared code freeze, and Replit's CEO called it "unacceptable and should never be possible" (Gizmodo, 2025).
Security, Governance, and Cost Control
LLM features open attack and liability surfaces that conventional application security checklists do not cover. Their running cost is also an architecture decision, not a line on a price list.
Security.
- The OWASP Top 10 for LLM Applications 2025 lists Prompt Injection (LLM01), Sensitive Information Disclosure (LLM02), Excessive Agency (LLM06), System Prompt Leakage (LLM07) and Unbounded Consumption (LLM10) among its risks.
- IBM's Cost of a Data Breach Report 2025, researched by the Ponemon Institute across 600 organisations, found that 97% of organisations that had an AI-related security incident lacked proper AI access controls, and 63% of the organisations studied had no AI governance policies (IBM, 2025).
Governance. This is a legal question, not only a technical one. In Moffatt v. Air Canada (2024 BCCRT 149), British Columbia's Civil Resolution Tribunal rejected Air Canada's argument that its website chatbot was responsible for its own statements, and held the airline liable for negligent misrepresentation after the bot gave a customer wrong information about bereavement fares (CTV News, McCarthy Tetrault). NIST's Generative AI Profile (NIST AI 600-1) is the standard reference for mapping these risks to controls.
Cost control. Epoch AI found that LLM inference prices for a given performance level have fallen between 9x and 900x per year depending on the task (Epoch AI, 2025), yet total spend still climbs as usage, context length and agent loops grow. Organisations are reacting: in the Stack Overflow Developer Survey 2026 results, 38% of respondents said their organisations set limits on usage, while 21% said their organisations are spending more.
What actually drives inference cost:
- Tokens per task: especially long contexts and retrieved chunks packed into prompts.
- Model choice and routing: whether easy requests go to smaller models.
- Agent loops: retries and reasoning tokens multiply calls per user action.
- Caching: of prompts and results, or the lack of it.
- Observability overhead: the cost of running evals and traces themselves.
- Unbounded consumption: OWASP treats it as a security risk because an attacker can run up the bill.
The Real Cost of Learning on the Job
Learning production AI on a live roadmap costs abandoned projects, budget overruns, rework and incidents, and primary data shows all four are common. The cost rarely shows up in a business case because it is spread across delays, tickets and risk rather than sitting on one line.
- Abandoned and over-budget projects. Pluralsight's 2026 Tech Skills Report, run with Wakefield Research among 1,500 tech executives, L&D leaders and practitioners in the US, UK and Australia, found that 69% of organisations abandoned tech projects due to a lack of tech skills, with cloud, cybersecurity and AI/ML projects suffering the most (Pluralsight, 2026). The same Pluralsight report found that 84% say skills gaps cause tech projects to go over budget at least some of the time. Those figures cover tech projects in general. For AI specifically, the 2025 Pluralsight finding above is the cleaner number.
- Training that does not transfer. Pluralsight's 2026 report also found that 77% of executives have seen an employee finish training or earn a certification and still be unable to do the job.
- A smaller learning budget than leaders assume. IBM found that only one third of enterprise AI developers are willing to invest more than two hours in learning a new AI development tool (IBM, 2025). Upskilling plans that assume engineers will self-teach evals and retrieval tuning on top of roadmap work rarely survive contact with a sprint.
- Rework. "Almost right" output is the top developer frustration in Stack Overflow's 2025 survey. In a team without evals, that output reaches production and comes back as tickets.
- Incidents and liability. The Air Canada ruling and the Replit database deletion are the two most-cited public examples. Neither required exotic technology. Both came from missing guardrails: no grounding or disclaimer on a customer-facing bot, and no hard separation between an agent and production data.
The opportunity cost nobody books. The engineers best positioned to learn production AI are usually the senior people already carrying the roadmap. Pulling them onto an AI learning curve slows the core product, and if the AI project then stalls, the organisation pays twice. A CFO-level model should include delay to existing roadmap items, rework, incident exposure and the probability of abandonment.
Three Ways to Close the Gap
There are three ways to give an existing software team production AI capability: upskill it, hire AI specialists into it, or partner with a team whose own senior engineers build inside your systems next to your engineers. Each fixes a different part of the gap, and most organisations end up combining them.
| Criterion | Upskill the existing team | Hire AI specialists | Partner to build inside your systems |
|---|---|---|---|
| What it fixes | Application-layer fluency, AI coding habits, shared vocabulary | Permanent in-house depth in one or two disciplines | Production disciplines applied from the first release, and a working example the team learns from |
| What it does not fix fast | Evals, retrieval tuning, security and cost controls, which are learned by shipping | A whole-team practice. One hire rarely changes how everyone else builds | Long-term headcount, which stays the company's decision |
| Time to production capability | Slowest. Depends on regular practice on real work | Gated by the hiring cycle for the most in-demand role | Shortest, when the partner's senior engineers start on the real system |
| Main risk | Training that does not transfer to the job | Retention, and a specialist isolated from the rest of the team | A partner that builds separately and hands over a black box |
| Who owns what gets built | The company | The company | The company, by contract |
| Best fit | AI as a feature, low data risk, flexible timeline | AI as a permanent core capability, with time to recruit | Production AI this year, sensitive data, or a team that should level up while shipping |
Upskill: Necessary in every scenario, sufficient in few. Gartner says generative AI will require 80% of the engineering workforce to upskill through 2027 (Gartner, 2024). There is no reliable published figure for how long a software team takes to reach production AI, so plan around what determines it:
- Frequency of use on real work. Microsoft's 2026 study found a 50.1% lift for engineers using agentic coding tools five or more days a week, against about 15% at three days a week (Microsoft, 2026).
- The practice the team already has. DORA's research suggests AI amplifies whatever practice exists, so strong testing, CI and observability habits give evals and monitoring something to build on.
- Learning on shipped work. Evals, retrieval tuning and cost controls are learned by shipping them, which is far easier next to someone who already practises them.
- Protected time. Courses stacked on a full sprint underperform, as the training-transfer and learning-budget findings above show.
The train-or-hire decision itself is covered in depth in AI upskilling vs hiring.
Hire AI specialists: The right answer when AI is a permanent core capability and the company can wait for the hire. The constraint is the market: in a Gartner survey of 300 US and UK organisations, 56% of software engineering leaders rated AI and machine learning engineers the most in-demand role for 2024, and rated "applying AI/ML to applications" the biggest skills gap (Gartner, 2024, reported by CDO Magazine). The cost drivers go beyond salary: recruiting time, competition for the role, ramp time and retention risk. A single specialist also meets DORA's "mirror and multiplier" effect, and one expert inside a team without eval habits tends to become a bottleneck rather than a standard-setter.
Partner to build inside your systems: This option puts production disciplines into the codebase from the first release while the existing team builds alongside. It only beats the other two when the partner's engineers work in the company's own repositories and systems, the company owns the result, and the internal team learns by building next to them. Consultancies and systems integrators that build in their own environment and hand over at the end leave the original gap open.
Assessment: Upskilling is the foundation in every scenario, hiring adds permanent depth, and partnering is strongest when production AI is needed before either of the others can mature. The common sequence is hybrid: partner for the first production systems while upskilling the team on that real work, then hire for the permanent roles the first year proves necessary. The hybrid AI team model covers how to structure that combination.
Which Path Fits Your Organisation
The answer to "do I need AI specialists or can my team do it?" depends on four things: what the AI has to do, how sensitive the data is, how soon it must reach production, and how mature the team's engineering practice already is. Match the path to the situation, not to the ambition.
- If AI is a feature inside an existing product, the data is low-risk and the timeline is flexible, upskill the team. The application layer is within reach, and DORA's research suggests AI amplifies existing practice, so strong testing, CI and observability habits carry over. Partnering is usually more than this situation needs.
- If AI is becoming a permanent core capability and leadership can wait for recruiting and ramp-up, hire specialists. A product built on the company's proprietary knowledge justifies permanent depth, provided there is a plan to spread the specialist's practice across the team rather than isolate it.
- If the roadmap needs production AI this year and a stalled pilot is expensive, partner to build inside your systems. Senior engineers who already practise evals, retrieval tuning and cost control start on the real system while the team learns on the same code, and the company owns everything that gets built.
- If the work touches customer data, regulated processes or production writes, get production depth in before launch. Security and governance have to be right the first time, and the Air Canada ruling shows that the company, not the chatbot, carries the liability.
- If the goal is to raise the standard of the whole engineering team, choose a path where learning happens on shipped work. That means partnering with engineers who build next to yours, or hiring with a deliberate plan to spread practice, not adding one isolated expert.
Before choosing, test the team against these questions:
- Can the team name the failure rate of its retrieval today?
- Is there an eval suite that runs on every prompt or model change?
- Who reviews traces of real user sessions, and how often?
- Are agent permissions scoped, and is production data unreachable from development environments?
- Is there a per-feature inference budget and a model routing policy?
- Is there a written AI governance policy?
Several "no" answers mean the team can build the demo but not yet the production system. For the team design that follows the decision, see AI-native team structure and how to build an AI-native engineering team.
How FutureProofing Builds Next to Your Team
FutureProofing builds AI products, operations automation and internal software with established companies, as partners. Its own senior engineers and teams do the work inside the partner's systems, next to the partner's engineers and experts, so the production layer gets built while the existing team learns it on its own codebase.
- The best engineers. FutureProofing hires the best engineers. Agentic coding tools such as Claude Code and Cursor are their default workflow, tested live in a paired AI challenge before anyone is hired, and Jess Mah personally runs the final technical interview. Most candidates do not make it through.
- Inside your systems, next to your experts. The work happens in the partner's own systems, so the partner's engineers see production disciplines applied to their own codebase. That is how the standard of the whole team rises.
- What gets built. AI products made from what a company knows, automation of the operations behind the business, and internal software that raises the standard of the whole team.
- Built for production, not the demo. Releases happen in stages with monitoring, and what gets built keeps improving after launch. That is the production reliability work where generalist teams stall.
- The partner owns it. What gets built is the partner's, and so is its data. Using a partner's knowledge does not mean training a model on it. Access, retention, provider terms and any proposed training are agreed with the partner.
- Governance before access. Mutual NDA and IP assignment come before any code or system access, with an MSA and SOW per project. The details are on the enterprise page.
- Scope. Partnerships run a few at a time, commercial terms are agreed per project, and long-term headcount stays the partner's decision.
An existing software team can ship the application layer. FutureProofing brings its own engineers and teams into your business to build the production layer inside your systems, so your team levels up while shipping and you own everything that gets built. That is what makes a business future-proof.
Tell us what you have in mind. A founder reads every proposal.
Collection · Build vs Outsource (decision)
