# If AI is so great, why isn't it working?
**作者**: vas
**日期**: 2026-04-30T01:16:39.000Z
**来源**: [https://x.com/vasuman/status/2049659161005470071](https://x.com/vasuman/status/2049659161005470071)
---
In the last 12 months, I've had the opportunity to chat with over 100 C-Suite executives at the largest companies across America. The conversation usually goes as follows: they spent a millions trying to bring AI into their business, whether in the form of new software licenses that promise to take work off their plate, or straight to the model providers in token spend.
Leadership buzzes non-stop about "going AI-first", yet when asked point-blank what has changed in the day-to-day, the answer is some version of nothing. The AP team is still doing AP the same way, month end close is still 22 days, reps are still hitting quota at 24%, and the CRM still has the same 30% data decay that it had in 2022.
I want to talk about why that is, why the models are not the problem, and what I've seen that makes AI actually work. For context I run Varick Agents, we specialize in bringing AI to enterprise to do real work without forcing massive migrations off systems of record.
## The models are good enough. Stop blaming the models.
First and foremost: the models are good enough. Stop blaming the models.
It's been a year and a half since GPT-o3, the first model where I could hand off a real task and not babysit the output too much. Then came Opus 4, GPT-5, and Gemini 3. Now we have Opus 4.7 and GPT 5.5. Pricing fell off a cliff, context windows expanded massively, and tool calling became much more accurate.
In the same window, harnesses became very robust and more scientific. Two years ago "harnesses" meant copy-pasting prompts off Twitter. Today it's recursive context engineering, eval pipelines, and tool stacks that genuinely stick. Anthropic wrote a canonical text on it (Building Effective Agents), and the punchline of that paper is "find the simplest solution possible, only increase complexity when needed." Most production agents are mostly code with one or two model calls in specific places. Boring but this is good, it works.
## So why isn't it working inside enterprise?
MIT NANDA's GenAI Divide report, August 2025: 5% of integrated AI pilots are pulling millions in value. The other 95% have nothing to show for it. BCG, Deloitte, RAND, IBM, Gartner, McKinsey all ran their own studies with their own methodologies and they all converge on the same number. BCG: 4% have hit AI value at scale. Deloitte: 6% achieve AI ROI within a year. RAND: 80%+ of AI projects fail, twice the rate of normal IT projects. IBM: 75% of AI initiatives haven't delivered expected ROI. McKinsey: 78% of orgs report regular AI use, 80%+ report zero EBIT impact.
That number has been flat through GPT-3, GPT-4, GPT-5, Claude 2 through 4.7, Gemini 1 through 3. Every model generation, the enterprise failure rate is identical. Despite models improving, failure rates have not come down. It turns out the model is not the bottle-neck.
## AI is working for one group: software engineers
AI is working for one group of people right now, at scale, because it's the group of people that rely the least on business logic. It's software engineers. The biggest winner from 18 months of AI improvement, by miles, has been engineers writing code in Cursor, Claude Code, Codex, etc. Some stats for you if for some reason you still don't believe in agentic engineering:
- GitHub's 2024 study clocked Copilot users at 55% faster on real tasks. 1 hour 11 minutes vs 2 hours 41 minutes on the same work.
- Anthropic ran an internal study in August 2025 across 132 engineers and 100,000 real Claude conversations. AI cuts developer task completion time by roughly 80%.
- Sundar Pichai said at the start of 2026 that 75% of new code at Google is AI-generated and engineer-approved. That number was 30% in April 2025.
Yes, the tools still overpromise on the hard stuff: security review, complex distributed systems, novel debugging. Caveat very real and noted. But the bread-and-butter productivity gain on shipping code is the biggest jump engineering has had since the IDE.
## Every other category vendors promised: flat metrics
Now look at every other category vendors have promised the same transformation.
1. Sales ops. Salesforce shipped Agentforce. Hubspot shipped Breeze. There are now ~100 "AI-native CRMs" in market all pitching some version of "the death of Salesforce." Has any of this moved actual sales numbers? Salesforce's own State of Sales report says reps spend 28% of their time selling. Pavilion 2024 says 24% of reps hit quota. Gartner says forecast accuracy is averaging 28%. Unchanged from 2022. The vendors have collectively raised billions for this and the metrics are flat.
2. Finance. Every accounting platform shipped some version of "AI for AP/AR/variance." Every "AI-native ERP" pitches close-cycle compression. The average finance team's close cycle has barely moved.
3. Marketing. Dozens of AI content tools. Marketers will tell you they save an hour a week. The marketing org's contribution to pipeline has not changed. If you made the unfortunate error of using AI-generated image/video for your marketing, you very likely got obliterated on social media.
4. Operations. Every workflow tool wrapped an LLM around its existing product. The ops team is doing the same job it was doing in 2023, and is arguably more bogged down than ever trying to wrangle AI products to fit their very unique business cases at the company.
## Why AI works for engineers and not anyone else
So why does AI work for engineers and not for any of these? What's different about engineers? As a former software engineer, engineering work has four properties that basically no other enterprise function has. Yes there are nuances but these are directionally correct, please relax in the comments.
- It's bounded. A function takes inputs and returns outputs. The scope of "fix this bug" lives inside a file or a module. The dependencies are explicit and importable.
- It's checkable. Compilers tell you in milliseconds whether the code parses. Tests tell you whether it works. Type systems catch entire classes of error before runtime. Feedback loop: seconds.
- The substrate is structured. Code lives in files, in version control, with a deterministic build pipeline underneath. Same input, same output. You can replay any state.
- The output is verifiable. A pull request is a discrete artifact. A reviewer can look at the diff in 10 minutes and say yes or no.
When you point a capable AI at work that's bounded, checkable, structured, and verifiable, the leverage is enormous. Cursor and Claude Code are the proof. And if we're being honest, the biggest reason is that the AI labs (OpenAI, Anthropic, Cursor) poured every single ounce of resources they had into figuring out software engineering. If they can make their own engineers better, they can make the models better, faster, and achieve the ever-elusive "AGI", which will then make every other task on the planet (Finance, Sales, Operations, Marketing, etc) much easier downstream.
But contrast software engineering with a finance close.
Finance involves AP, AR, intercompany reconciliations, FX, accruals, journal entries, and exception handling that spans NetSuite, Concur, three banks, two ERPs from acquisitions, a custom intake form, and a Slack channel where the controller flags "weird stuff she sees." The "process" is documented in an SOP that doesn't match what actually happens. The output is "the close was clean," which takes two senior accountants two days to verify.
Sales ops involves a CRM, an outbound tool, a calendar, a notes platform, an enrichment vendor, an attribution tool, and a Slack channel where the AE is asking the CRO whether to discount this deal. None of those systems share state cleanly. The process for qualifying a lead is different across reps, even on the same team.
This is what every ops function looks like in every company Varick has ever audited. None of it is bounded, checkable, structured, or verifiable the way code is. And trying to wrangle generic AI to these functions that are incredibly specific to your company and its processes is a fools errand.
Pointing an LLM at this work gives you negative ROI. The operator was doing the work in 30 minutes. Now they're doing the work in 30 minutes plus another 30 minutes correcting the AI's mistakes. Most if not every vendors' "AI for [department]" has the same arc. A nice flashy demo showing how great it works for startups, then a big series A, then quietly killed after it fails to work for enterprise.
## The four failure modes I see every time
I've spent the last 18 months deploying AI agents inside enterprise teams across finance, sales, ops, engineering, and marketing, at companies doing $500M to $5B+ in revenue. Every failure I've seen, including the ones we caught early, traces back to the same four things.
## 1. They skip the audit
The single biggest predictor that an AI pilot is going to fail: the team starts building before they understand the workflow they're supposedly automating. Every company is so unique that simply duplicating and agent that worked for one company onto another is not going to work.
Most commonly, companies look at the most obvious workflow, scope it from the SOP, build for that, ship it, demo it to the CFO, deploy. And then it collides with the actual workflow, which is different from the documented one in 50 places.
The actual workflow always includes things the SOP doesn't mention. The "I always check this spreadsheet first" step. The "I email Sarah directly because the system notification doesn't work" step. The 17 exception types the team handles every month. The unwritten rule that anything over $5M loops in the controller, even though the threshold says $10M.
When you build for the documented process, you automate 70% of the volume and break on 30%. The 30% that breaks creates more work for the team than they had before, because now they have to fix the AI's mistakes on top of doing the work.
The audit is the part where you sit with the people doing the work, watch them do it, and map what actually happens. We do this for at least four weeks if not longer before touching a model. It's the medicine that no one wants to take but you have to to prep for surgery, which is what AI implementations are.
We call the gap between SOP and actual workflow the conformance gap. In typical engagements we see 30%+. In heavy exception-driven workflows like AP exception handling or supply chain disruption response, we routinely see over 70%.
The reason most AI pilots skip the audit is that it doesn't feel like AI, it instead feels like operational consulting. Most AI vendors won't do it because it's not their core skill. Most internal teams don't do it because they "already know the workflow." They don't. Every time a client has come to us saying "we know the problem and we want you to build X", we end up performing an audit anyways and by the end of it they've been convinced otherwise.
## 2. They throw everything at the LLMs and expect it to work
The second biggest predictor: using the LLM for parts of the workflow that don't need an LLM.
LLMs are seductive. Once you have one, every problem looks LLM-shaped. Need to extract a value from a document? Ask the model. Compare two values? Ask the model. Route a result based on a number? Ask the model. The team builds an architecture that's 90% LLM calls and 10% code. The system is slow and incredibly expensive, while simultaneously hallucinating 10% of the time, which is fine for a chat interface and unacceptable for AP automation. The CFO kills it.
Production systems that actually work look almost boring. They're 85% code and 15% LLM. The LLM goes where judgment lives: pulling structured data out of an unstructured invoice, classifying an exception into one of 31 known patterns, drafting an explanation for a human reviewer. The rest is database queries, comparisons, deterministic logic, branch routing.
This compounds with the audit problem. Teams that didn't audit don't know which parts of the work are pattern-matchable and which need real judgment. So they default to the LLM for everything and ship something that breaks.
## 3. Agent sprawl
Arguably the most silent but deadly problem in the AI implementation space: this one only shows up after month 6. By the time you notice, you've already spent the budget and the time.
Sprawl is a person-level problem before it's a department-level problem. Every individual employee with AI access turns into their own agent factory.
For example, Sarah in AP builds her own agent in Lovable to classify invoices. The controller spins up a separate one to reconcile intercompany transfers. The FP&A lead vibe-codes a workflow to pull variance reports. The CRO's chief of staff has a personal agent that summarizes Salesforce notes before every QBR. The marketing manager built a content agent. The recruiting coordinator built a candidate-screening agent. So on and so forth.
If you were to multiply that across a 200-person operations org you'd end up with 50 to 100+ separate AI workflows running across the business, each one built by a different person, each one with its own quirks, each one solving a similar 3 or 4 problems in 7 different ways.
Even in the best case scenario where every single one of these agents actually works, you end up with 100 disconnected systems, each with its own ingestion pipeline, its own approval logic, its own logging, its own model config, and its own prompts. There is no common agent spine, no shared memory, and no shared knowledge of how the company actually runs. Marketing's content agent has zero awareness that customer support is currently dealing with 50 tickets about the exact thing it's writing copy about. Finance's invoice agent has no idea that procurement just blacklisted that vendor last week.
Finally, the inevitable happens: a model gets deprecated, or an API endpoint changes, or an employee leaves, or a vendor pushes a breaking change. Now you have 50 of those 100 agents breaking in production, no one is on call for any of them (because building can be done by vibe-coders but fixing cannot), and the people who originally built them are either too busy or no longer at the company. The CTO's engineering team is suddenly spending half its time playing janitor for AI workflows nobody actually owns, and no one even knows what is worth fixing vs killing off.
The downstream cost is astronomical. Engineering maintainability is a literal nightmare, since every agent is its own bespoke project with its own bugs and its own deploy story. Security is even worse, with every personal agent carrying its own API keys, its own data access, and its own potential for data exfiltration. Compliance and audit logging are basically nonexistent across the fleet, and don't even ask about governance. A real example we've all heard of: the legal team finds out three months later that someone's personal marketing agent has been blasting customer data into a third-party LLM API that wasn't on the approved vendor list. The AI that was meant to bring you 'untold ROI' is now costing you 10s of millions.
The fix has to be architectural and it has to be planned out from day one. You need a single orchestration layer that sits on top of the existing software stack, with shared infrastructure for ingestion, approvals, audit logging, model routing, and knowledge. Every new use case from any person or any process lands as configuration on top of that single platform, and no more bespoke vibe-coded side projects that nobody else in the company even knows exist. Ask your engineers how important this is, if you dont' believe me.
Once you actually have the platform, the economics compound dramatically. The first agent on the platform takes 12 weeks but the next takes 9 and the third takes 4. Without the platform, every single agent costs roughly the same to build, and the integration debt eventually consumes the entire AI budget.
This is the part most CTOs underestimate, and it's also the part that most "AI vendors" can't help you with at all. They're selling you one SKU, and that SKU has zero awareness of the other 7 you're already running, let alone the 80 personal agents your team built on weekends.
## 4. They treat AI as a side-project instead of infrastructure
The fourth failure mode is the slowest, the hardest for boards to spot, and quietly the most expensive over time.
Most companies budget AI initiatives like any other software project: plan, build, ship, declare victory, move on. That logic works for traditional software because once you build it, it stays built. AI is the opposite. Every quarter, something underneath you shifts: for example, a new release is dramatically better at your specific workload, or worse, the model you depended on quietly gets distilled and degrades. Anthropic alone has retired roughly 9 models in 18 months, OpenAI has retired even more, pricing changes on a quarterly basis, and rate limits get tightened with no warning.
A real example: in April 2026 Anthropic publicly acknowledged that engineering errors had degraded Claude Code performance for over a month, and that paid Max subscribers ($200/month) were hitting their quota in 19 minutes instead of the advertised 5 hours. They apologized publicly, but the production workloads built on those guarantees had already broken in the meantime.
The AI world moves faster than you can ever keep up with, and has no respect for your beautiful 6-month project plan. The deployments that actually pay off treat AI as continuously evolving infrastructure with a dedicated team that owns ongoing optimization. They monitor quality, swap models when better ones ship, retire agents that have stopped earning their keep, and renegotiate with vendors when something breaks underneath them. Most importantly, they avoid vendor lock in as much as humanly possible. If your entire AI initiative relies on Anthropic, you're stuck no matter what models get degrated, quantized, price-hiked, rate limited, etc.
## What the 5% that ships and stays in production does consistently
Ok so what does the 5% that ships and stays in production do consistently that makes them so good:
1. They audit before they build. Four weeks (often longer) of mapping the actual workflow before anyone touches a model. The audit produces a digital twin: a live map of how work moves through the org, where the conformance gaps are, what's pattern-matchable, and what genuinely needs human judgment. The document itself matters less than the alignment it forces between the AI team and the operators. Make sure everyone is aligned on what the bottle-necks are, what the optimal state should be, and what is going to be done to fix it.
2. They decompose the work until most of it is deterministic. LLM goes ONLY where judgment is absolutely required, while plain code goes everywhere else. Most production systems we ship at Varick end up as 5-10 deterministic steps with maybe one or two model calls in specific places. Boring in production is genuinely the goal, and is how we've seen the most success.
3. They build a single orchestration layer that sits on top of the existing software stack. At Varick, we call this the single pane of glass. Finance, sales, ops, and engineering agents all live on the same platform, share the same context, and can talk to each other when they need to. Every new use case lands as configuration on top of the platform. In turn, sprawl is dead on arrival.
4. They stay model-agnostic. Abstractions get built at the task level, not at the model level. Each step routes to the best-fit model at any given moment. When OpenAI deprecates a model or Anthropic ships something dramatically better, the routing layer absorbs the change and your workflow keeps running without anyone noticing.
5. They treat the deployment as continuously evolving infrastructure. There is a real team responsible for ongoing tuning, retiring agents that aren't earning their keep anymore, and shipping improvements every quarter. The deployments that pay off over five years are the ones that get tuned every quarter if not every month, not the ones declared "done" at go-live. You have to get over this fact if you want to succeed with AI.
## The labs are quietly conceding the same point
The labs are quietly conceding the same point Every model lab has now figured out that selling the model alone isn't enough. Selling the workflow runtime alone isn't enough either. You need the model, plus the runtime, plus a team that goes and embeds inside the enterprise and figures out what to actually build for it.
In other words, the labs are quietly conceding the entire thesis of this article. Process is the bottleneck. It's been the bottleneck the whole time, and unless something dramatic changes, it's going to stay the bottleneck.
The interesting question is who actually delivers the process work. Big consulting firms have the relationships and the headcount but historically ship strategy decks far more often than production agents. The labs have the best models and the best marketing but they don't embed inside your operations. The thing that genuinely works is some hybrid: production-grade AI combined with serious operational fieldwork, where someone audits first, decomposes the work, builds, runs, and tunes.
## What I'd do if I had to start from scratch
If a CEO walked into our office tomorrow with $1M and 6 months and said "deploy AI in our enterprise, please don't fail", here's the path I would run.
Month 1: audit. Embed with the operators in whatever function we're targeting and watch them actually do the work end-to-end. Map the conformance gap between the SOP and reality. Figure out which workflows are pattern-matching and which ones need real human judgment. Produce a digital twin we can build against in month 2.
Month 2: architect the highest-leverage workflow. Decide what's pattern vs judgment, what's deterministic vs probabilistic, and pick the right model for each step (mostly small fast cheap models, with a frontier model only on the steps that genuinely need it). Define the human-in-the-loop touchpoints and the audit logs.
Months 2.5-4: build. Soft launch in production where humans approve every action, watch the agent learn from corrections, track accuracy daily, and fix what breaks before going wider.
Month 4-6: go live. Cut over with humans still in the loop on the high-stakes decisions, and watch the metrics that actually matter (cycle time, error rate, throughput) for at least two weeks before anyone declares a win.
Month 6: stand up the continuous-tuning function. The model market will change next quarter. Your workflow will change next quarter. The optimization team is what keeps the agent useful through both.
By the end of those 6 months you have one workflow live in production with real ROI in the books, a platform that can absorb the next workflow in 8 weeks instead of 24, and a tuning team that keeps the whole thing alive going forward.
## The "models got smart" chapter is over
The "models got smart" chapter is already over. We had a glorious run but enterprise should no longer be holding out for AGI. The time to take action is now. And the next decade belongs to the companies that build the operational layer underneath the models, as opposed to the ones who spend another five years pouring frontier AI onto a mess of systems and wondering why nothing has actually changed.
This is exactly what we do at Varick. Taking on new clients doing over $2B in revenue starting June 1, 2026. Check us out at varickagents.com. Even better - we're hiring. If you're talented, come meet the team.
## 相关链接
- [vas](https://x.com/vasuman)
- [@vasuman](https://x.com/vasuman)
- [269K](https://x.com/vasuman/status/2049659161005470071/analytics)
- [$500M](https://x.com/search?q=%24500M&src=cashtag_click)
- [$5B](https://x.com/search?q=%245B&src=cashtag_click)
- [$5M](https://x.com/search?q=%245M&src=cashtag_click)
- [$10M](https://x.com/search?q=%2410M&src=cashtag_click)
- [$1M](https://x.com/search?q=%241M&src=cashtag_click)
- [$2B](https://x.com/search?q=%242B&src=cashtag_click)
- [varickagents.com](https://varickagents.com/)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [9:16 AM · Apr 30, 2026](https://x.com/vasuman/status/2049659161005470071)
- [269.5K Views](https://x.com/vasuman/status/2049659161005470071/analytics)
- [View quotes](https://x.com/vasuman/status/2049659161005470071/quotes)
---
*导出时间: 2026/5/1 21:55:04*
---
## 中文翻译
# 如果 AI 这么厉害，为什么还没用起来？
**作者**: vas
**日期**: 2026-04-30T01:16:39.000Z
**来源**: [https://x.com/vasuman/status/2049659161005470071](https://x.com/vasuman/status/2049659161005470071)
---
在过去 12 个月里，我有机会与美国 100 多家最大企业的高管层进行了交流。对话通常是这样的：他们花了数百万美元试图将 AI 引入业务，无论是购买承诺能替他们分担工作的新软件许可证，还是直接向模型提供商支付 Token 费用。
领导层整天都在谈论“AI 优先”，但当被直接问及日常工作发生了什么变化时，答案无一例外是“没什么变化”。应付账款（AP）团队仍在以老一套方式处理 AP，月度结账仍需 22 天，销售代表的配额达成率仍只有 24%，CRM（客户关系管理）系统的数据腐烂率仍和 2022 年一样维持在 30%。
我想谈谈为什么会这样，为什么模型不是问题所在，以及我亲眼看到的能让 AI 真正起作用的方法。作为背景，我经营着 Varick Agents，我们专注于将 AI 引入企业以完成实际工作，而无需强制大规模迁移现有记录系统。
## 模型已经够好了。别再怪模型了。
首先是：模型已经够好了。别再怪模型了。
距离 GPT-o3 发布已经过去一年半了，那是第一个我可以交付真实任务而不必过度监督其输出的模型。随后出现了 Opus 4、GPT-5 和 Gemini 3。现在我们有了 Opus 4.7 和 GPT 5.5。价格暴跌，上下文窗口大幅扩展，工具调用的准确性也大大提高。
在同一时期，AI 的控制框架变得非常稳健和科学。两年前，“控制框架”意味着从推特上复制粘贴提示词。今天则是递归上下文工程、评估管道，以及真正稳固的工具栈。Anthropic 写了一篇关于此的权威文章（《构建有效的智能体》），该文章的要点是“找到尽可能简单的解决方案，仅在必要时增加复杂性”。大多数生产环境中的智能体本质上主要是代码，仅在特定位置有一两次模型调用。这很枯燥，但这是好事，因为它有效。
## 那为什么它在企业内部不起作用？
MIT NANDA 2025 年 8 月的《GenAI 差距》报告显示：5% 的集成 AI 试点项目创造了数百万美元的价值。其余 95% 一无所获。BCG、Deloitte、RAND、IBM、Gartner 和 McKinsey 都使用各自的方法进行了研究，它们都收敛于同一个数字。BCG：4% 实现了大规模 AI 价值。Deloitte：6% 在一年内实现了 AI 投资回报率（ROI）。RAND：80% 以上的 AI 项目失败，是普通 IT 项目失败率的两倍。IBM：75% 的 AI 计划未能交付预期的 ROI。McKinsey：78% 的组织报告定期使用 AI，但 80% 以上报告息税前利润（EBIT）影响为零。
这一数字在 GPT-3、GPT-4、GPT-5、Claude 2 到 4.7、Gemini 1 到 3 的整个时期一直保持平稳。每一代模型，企业失败率都是一样的。尽管模型在改进，失败率并没有下降。事实证明，模型不是瓶颈。
## AI 只对一类人有效：软件工程师
AI 目前只对一类人有效、且能大规模应用，因为这群人最不依赖业务逻辑。他们是软件工程师。过去 18 个月 AI 进步的最大受益者，无疑是那些在 Cursor、Claude Code、Codex 等工具中编写代码的工程师。如果你因为某种原因仍然不相信“智能体式工程”，这里有一些数据：
- GitHub 2024 年的研究显示，Copilot 用户在真实任务上的速度提高了 55%。同样的工作，1 小时 11 分钟 vs 2 小时 41 分钟。
- Anthropic 在 2025 年 8 月进行了一项内部研究，涉及 132 名工程师和 10 万次真实的 Claude 对话。AI 将开发人员的任务完成时间缩短了约 80%。
- Sundar Pichai 在 2026 年初表示，谷歌 75% 的新代码是由 AI 生成并经工程师批准的。而在 2025 年 4 月，这一数字还是 30%。
是的，这些工具在处理难题时仍然过度承诺：安全审查、复杂的分布式系统、新颖的调试。这些警告非常真实，值得注意。但在交付代码方面的生产力提升，是工程领域自 IDE（集成开发环境）以来最大的飞跃。
## 供应商承诺的所有其他类别：指标持平
现在看看供应商承诺了同样变革的所有其他类别。
1. **销售运营**。Salesforce 推出了 Agentforce。Hubspot 推出了 Breeze。现在市场上大约有 100 家“AI 原生 CRM”，都在推销某种版本的“Salesforce 之死”。这些真的提升了实际销售数字吗？Salesforce 自己的《销售状况报告》称，销售人员将 28% 的时间花在销售上。Pavilion 2024 的数据显示 24% 的代表达到了配额。Gartner 称预测准确率平均为 28%。与 2022 年相比没有变化。供应商们为此筹集了数十亿美元，但指标却持平。
2. **财务**。每个会计平台都推出了某种版本的“用于 AP/AR/差异分析的 AI”。每个“AI 原生 ERP”都在推销缩短结账周期。普通财务团队的结账周期几乎没有变化。
3. **市场营销**。出现了数十个 AI 内容工具。营销人员会告诉你他们每周节省一小时。但营销组织对销售管道的贡献并没有改变。如果你不幸在营销中使用了 AI 生成的图像/视频，你很可能会在社交媒体上被喷得体无完肤。
4. **运营**。每个工作流工具都在其现有产品周围封装了一个大语言模型（LLM）。运营团队仍在做着 2023 年所做的同样的工作，而且可以说比以往任何时候都更加陷入困境，试图让 AI 产品适应公司非常独特的业务案例。
## 为什么 AI 对工程师有效而对其他人无效
那么为什么 AI 对工程师有效，而对其他人却无效？工程师有什么不同？作为一名前软件工程师，工程工作具有基本上没有其他企业职能具备的四个属性。是的，这其中有细微差别，但这些方向是正确的，请在评论中保持轻松。
- **它是封闭的**。函数接收输入并返回输出。“修复这个 Bug”的范围位于一个文件或模块内。依赖项是显式的且可导入的。
- **它是可检查的**。编译器会在毫秒级告诉你代码是否可解析。测试告诉你它是否有效。类型系统在运行前捕获整类错误。反馈循环：秒级。
- **基质是结构化的**。代码存在于文件中，存在于版本控制中，下面有一个确定性的构建管道。相同的输入，相同的输出。你可以重放任何状态。
- **输出是可验证的**。拉取请求是一个离散的工件。审查者可以在 10 分钟内查看差异并说是或否。
当你将一个强大的 AI 指向一项封闭、可检查、结构化且可验证的工作时，杠杆效应是巨大的。Cursor 和 Claude Code 就是证明。坦率地说，最大的原因是 AI 实验室投入了所有资源来搞清楚软件工程。如果他们能让自己的工程师变得更优秀，他们就能让模型变得更好、更快，并实现那个难以捉摸的“AGI”（通用人工智能），这将使地球上所有其他任务（财务、销售、运营、市场营销等）在下游变得容易得多。
但是，将软件工程与财务结账对比一下。
财务涉及 AP、AR、公司间对账、外汇（FX）、应计项目、日记账分录，以及跨越 NetSuite、Concur、三家银行、两个来自收购的 ERP、一个自定义录入表单以及一个 Slack 频道的异常处理——在 Slack 频道里，主计长会标记她看到的“奇怪东西”。“流程”记录在 SOP（标准作业程序）中，但与实际发生的情况并不匹配。输出是“结账很干净”，这需要两名高级会计师花两天时间来验证。
销售运营涉及 CRM、外联工具、日历、笔记平台、数据增强供应商、归因工具，以及一个 Slack 频道——在频道里 AE（客户经理）询问 CRO（首席营收官）是否应该给这笔交易打折。这些系统没有一个能干净地共享状态。合格线索的流程在不同的代表之间是不同的，即使在同一个团队也是如此。
这就是 Varick 审计过的每家公司的每个运营职能的样子。没有一个像代码那样是封闭的、可检查的、结构化的或可验证的。试图让通用 AI 去适应这些对贵公司及其流程极其具体的职能，是一件愚蠢的差事。
让 LLM 处理这种工作会带来负 ROI。操作员本来 30 分钟就能完成工作。现在他们要花 30 分钟做工作，外加 30 分钟纠正 AI 的错误。大多数（如果不是每一个）供应商的“面向[某部门]的 AI”都有同样的轨迹：先是展示其对初创公司多么有效的精彩演示，然后完成 A 轮巨额融资，最后因无法在企业中发挥作用而被悄然关停。
## 我每次看到的四种失败模式
在过去 18 个月里，我一直在收入规模从 5 亿美元到 50 亿美元以上的公司的财务、销售、运营、工程和市场营销团队中部署 AI 智能体。我看到的每一次失败，包括我们早期发现的那些，都可以追溯到同样四件事。
## 1. 他们跳过了审计
预测 AI 试点项目失败的最大单一指标：团队在理解他们 supposedly 要自动化的工作流程之前就开始构建了。每家公司都是如此独特，以至于简单地将一家公司成功的智能体复制到另一家公司是行不通的。
最常见的情况是，公司查看最明显的工作流程，根据 SOP 进行范围界定，为此构建，发布，向 CFO 演示，部署。然后它与实际工作流程发生冲突，而实际流程在 50 个地方都与文档记录的不同。
实际工作流程总是包含 SOP 中未提及的内容。“我总是先检查这个电子表格”的步骤。“我直接给 Sarah 发邮件，因为系统通知不起作用”的步骤。团队每月处理的 17 种异常类型。不成文的规定是，任何超过 500 万美元的事情都要让主计长参与，即使阈值写着 1000 万美元。
当你为文档化的流程构建时，你自动化了 70% 的体量，但在 30% 上崩溃。崩溃的 30% 给团队创造了比以前更多的工作，因为他们现在必须在完成工作之上修复 AI 的错误。
审计就是你要和做工作的人坐在一起，看他们做，并映射实际发生的事情。我们在接触模型之前会这样做至少四周，甚至更长。这是没人想吃的药，但为了给手术做准备必须吃，而 AI 实施就是手术。
我们将 SOP 和实际工作流程之间的差距称为**一致性差距**。在典型的项目中，我们看到这一差距超过 30%。在重度异常驱动的工作流程中，如 AP 异常处理或供应链中断响应，我们 routinely 看到超过 70% 的差距。
大多数 AI 试点项目跳过审计的原因是，它感觉不像 AI，而感觉像运营咨询。大多数 AI 供应商不会这样做，因为这不是他们的核心技能。大多数内部团队不这样做，因为他们“已经知道工作流程”。他们并不知道。每次客户来找我们说“我们知道问题，希望你们构建 X”时，我们最终还是会进行审计，最后他们都被说服了。
## 2. 他们把所有东西都扔给 LLM 并指望它起作用
第二大预测指标：将 LLM 用于工作流程中不需要 LLM 的部分。
LLM 很诱人。一旦你拥有了一个，每个问题看起来都像是 LLM 形状的。需要从文档中提取值？问模型。比较两个值？问模型。基于数字路由结果？问模型。团队构建了一个 90% 是 LLM 调用和 10% 是代码的架构。系统缓慢且极其昂贵，同时还有 10% 的时间产生幻觉，这对聊天界面来说没问题，对于 AP 自动化来说则是不可接受的。CFO 会扼杀它。
真正有效的生产系统看起来几乎很无聊。它们是 85% 的代码和 15% 的 LLM。LLM 用于需要判断的地方：从非结构化发票中提取结构化数据，将异常归类为 31 种已知模式中的一种，为人工审查者起草解释。其余的是数据库查询、比较、确定性逻辑和分支路由。
这与审计问题复合了。没有进行审计的团队不知道工作的哪些部分可以模式匹配，哪些需要真正的判断。所以他们对所有事情默认使用 LLM，并发布一个会崩溃的东西。
## 3. 智能体蔓延
可以说这是 AI 实施领域中最无声但致命的问题：这只在第 6 个月后才显现。当你注意到时，你已经花完了预算和时间。
在成为部门级别的问题之前，蔓延首先是一个人的问题。每个拥有 AI 访问权限的员工都变成了他们自己的智能体工厂。
例如，AP 团队的 Sarah 在 Lovable 中构建了自己的智能体来分类发票。主计长启动了一个单独的智能体来协调公司间转账。FP&A 负责人用 vibe-coding 编写了一个工作流来提取差异报告。CRO 的幕僚长有一个个人智能体，可以在每次季度业务回顾（QBR）之前总结 Salesforce 笔记。营销经理构建了一个内容智能体。招聘协调员构建了一个候选人筛选智能体。以此类推。
如果你在一个 200 人的运营组织中乘以这种情况，最终会在整个业务中运行 50 到 100 多个独立的 AI 工作流，每一个都是由不同的人构建的，每一个都有自己的怪癖，每一个都以 7 种不同的方式解决类似的 3 或 4 个问题。
即使在最好的情况下，即每一个智能体真的都能工作，你最终也会得到 100 个断开的系统，每个都有自己的摄取管道、自己的批准逻辑、自己的日志记录、自己的模型配置和自己的提示词。没有共同的智能体脊柱，没有共享的记忆，也没有关于公司实际如何运行的共享知识。营销的内容智能体完全不知道客户支持目前正在处理 50 张关于它正在撰写的文案确切内容的工单。财务的发票智能体不知道采购部门上周刚将那个供应商列入黑名单。
最后，不可避免的事情发生了：模型被弃用，或 API 端点发生变化，或员工离职，或供应商推送了破坏性更改。现在你有 100 个智能体中的 50 个在生产环境中崩溃，没有任何人待命（因为构建可以由 vibe-coder 完成，但修复不能），而最初构建它们的人要么太忙，要么已经离开了公司。CTO 的工程团队突然花了一半时间充当没人真正拥有的 AI 工作流的清洁工，甚至没人知道什么值得修复，什么值得扼杀。
下游成本是天文数字。工程可维护性简直是一场噩梦，因为每个智能体都是自己的定制项目，有自己的错误和自己的部署故事。安全性更糟，每个个人智能体都有自己的 API 密钥、自己的数据访问权限和自己的数据渗漏潜力。合规性和审计日志在整个集群中基本不存在，甚至别问治理问题了。我们都听说过一个真实的例子：法律团队在三个月后发现，某个人的个人营销智能体一直在将客户数据轰进未经批准的供应商名单之外的第三方 LLM API。原本旨在为你带来“不可估量的 ROI”的 AI，现在正在让你损失数千万美元。
解决方案必须是架构性的，必须从第一天就开始规划。你需要一个位于现有软件堆栈之上的单一编排层，拥有用于摄取、批准、审计日志记录、模型路由和知识的共享基础设施。来自任何人或任何流程的每个新用例都作为配置添加到该单一平台之上，不再有公司里其他人都不知道存在的定制 vibe-coded 侧面项目。如果你不相信我，问问你的工程师这有多重要。
一旦你真正拥有了平台，经济……