# How we built our automatic LLM training infrastructure
**作者**: Amar Singh
**日期**: 2026-05-26T00:23:30.000Z
**来源**: [https://x.com/AmarSVS/status/2059067870743875684](https://x.com/AmarSVS/status/2059067870743875684)
---
We had a vision of a fully integrated end-to-end LLM training platform. We wanted to make it dead easy for a user to train and deploy small models that get better over time. You route your inferences through our platform, build a dataset, then train and deploy a model. Afterwards, we continue collecting your inferences to make follow-up improvements to the model.
It’s a simple pitch, but building it was harder than we expected.
I’m going to talk about how we got the “Train” button to work. This involved a whole host of issues that usually live in the collective mindshare and bash scripts of labs and researchers:
- The dataset needs to be in the right shape and the hyperparameters must fit the dataset.
- You must select the right model and training configuration.
- GPUs need to be available.
- The runtime must support the chosen model family, with the correct kernels, parallelism and checkpoint format.
- Your evals have to measure something that matters.
- When the training is done, your trained model needs to be one click from deployment.
- A whole bunch of niche edge cases that might arise in the whole process.
These are all barriers that compound to make the whole pipeline difficult to build. This post is meant to talk about how we tackled and solved all of this.
## Data in, dataset out
Before training anything, we need a dataset. Customers get data into Catalyst one of three ways: they route their inference calls through our proxy, they upload a file containing their inferences that were collected elsewhere, or they send agent traces through our OTEL-compatible Catalyst Tracing SDK.
We have to normalize these samples such that they can be valid inputs to our pipeline. This means that we have to address different formatting issues like Anthropic putting base64 images inside a source object, OpenAI using image_url, and the mess that Gemini puts up through their API. We handle all of this at ingest so that we can continue on into the trainer.
After that, there is a data preprocessing pipeline that is in charge of cleaning the dataset. We drop empty messages and excessively long samples, and remove evaluation set data points from the train set. There are a few more steps, but the point of this is to have confidence in what we feed into the training loops.
Recently, we have also added HALO to handle larger agentic datasets, as traces can be particularly hard to analyze. We look for semantic patterns that might be overlooked like model refusals or formatting errors. In some cases, we want to do rejection sampling such that we can set a higher bar for the model that we train.
These steps are mostly common practice, but many of these have been added after lessons from training models manually. This pipeline has been built over many iterations through the last year.
## We pick your hyperparameters for you
If you are familiar with training models, you know that there is a large subset of hyperparameters that can be tweaked. This has been one of the hardest things to get right, especially because of all the niche errors that arise with misconfigurations, but we have an opinionated system to figure out these values.
When you click “train”, we start an analysis step over the dataset. We look at inputs like input/output character statistics, semantic matter, modalities, and other features. Based on this, we have a complex decision tree of which parameters to enable to optimize the training run for both speed and accuracy. We adjust things like sample packing, parallelisms, learning rates, epoch lengths, batch sizes and gradient accumulations, vision towers and what parameters get frozen, and more.
On top of that we have a recipe layer, our curated starting points. A recipe gives us the base model, the judge model, the LoRA topology, the FSDP version, the learning rate, and the eval cadence. We add your dataset’s statistics on top, tweaking the recipe to ensure we have appropriate settings.
When running evals, we size the sandbox based on the number of model parameters and evaluation size. Larger runs get more GPUs, defining larger based on either the model size or the number of samples to process. We also support routing the evaluations to different GPU targets. Per-GPU memory utilization, max concurrent sequences, generation batch size, and worker thread counts get filled in from a table keyed on whatever GPU type the eval is running on. We rigorously test different hardware types to make sure that there is no model drift because of it.
We can also override and manually configure these parameters if we ever think something should be tweaked.
## Wherever there’s compute
Training jobs need GPUs, we needed to decide which ones and where to get them.
We use our own GPU cluster as our primary training pool, as that gives us a layer of predictability and control. When we need excess capacity we fall back to serverless GPU providers, as without this customers would experience long queue times during peak load. We needed to strike a balance here, as while the cluster is cheaper per GPU hour and more reliable, we didn’t want to waste capacity during usage troughs. On the other hand, if we always trained on serverless we would absorb higher GPU costs and higher error rates. We built a scheduler that walks a priority ladder, checking availability from each provider and assigning the job in seconds when capacity is there.
Since we are running across multiple providers we couldn’t tie our runtime to a single target, and so our container is provider-agnostic. It takes a job config as CLI arguments and a payload, and it reports back over signed webhooks. No orchestration logic lives in the training container.
An interesting consequence comes from the time limits that serverless providers impose. Most cap how long a sandbox can run before they are pre-empted. Twenty-four hours is common, and serious fine-tunes on a large model take longer than that.
Our solution was rollover via checkpointing. Periodically, the runtime uploads a fully resumable checkpoint to durable storage. About an hour before the sandbox gets pre-empted, our control plane launches a new training job from the most recent checkpoint and stops the old one. From the customer’s perspective, there is one continuous job. On the infrastructure side, that job may have existed in many places over its lifetime.
This allows us to run multi-day fine-tunes on infrastructure where no sandbox lives that long. It also doubles as a recovery mechanism for when something does go wrong. If there’s a runtime crash, NaN loss, node failure, etc., the job gets picked up from its last resumable checkpoint and keeps going.
## The training runtime
A few months in we picked our training runtime stack and committed to it. We didn’t realize it at the time, but that turned out to be one of the most consequential decisions we made.
We started with the Megatron-Bridge and Megatron-Core ecosystem. On paper it was obvious, Megatron had the most mature parallelism story for very large models. The kernels were battle tested, and the lineage includes some of the most ambitious training runs in the field. We got it working and started running jobs on it.
We quickly started to run into hard walls, however. Adding new models was a nightmare and the training speeds themselves were pretty slow, especially on sub-100B models. We also could not benefit from the larger kernel ecosystem since most of these patches required large engineering migrations. We needed something where we could iterate and upgrade faster.
To solve that, we moved to our custom framework on top of PyTorch and Hugging Face Trainer. We patch with our implementations of parallelism and kernels so we can benefit from the fast-moving frontier. The downside was that parallelisms got much more complex, especially with long context or large parameter models. We created our own network of parallelism interactions using Ring/Ulysses Attention, FSDP2, and some custom NCCL operations.
This also meant that we had more fine grained control over adding kernels. We integrated ScatterMoE/SonicMoE to the MoE models that we had in our arsenal. We were able to speed up trainings by an order of magnitude, to the point where we had faster training iteration steps than some Megatron Core loops themselves.
We also had to account for custom loss functions through these parallelism settings, like calculating the loss values when the context is sharded or masking through sample packing. With models having different ways to process multimodal masking, we had to add custom model-specific image/video pathways so that we could train with them effectively as well.
Some other considerations:
Frozen vision tower pre-sharding under FSDP2. When you fine-tune a vision-language model, you usually want to keep the vision tower frozen. It’s expensive to train and the downstream task is usually a language task anyway. FSDP2 doesn’t natively handle a module tree where some subtrees are frozen and others are trainable, as the root wrap groups them together and the gradient bookkeeping breaks. We solved this by detecting the frozen vision case during model loading and sharding the vision subtree first, before the recursive wrap reaches it. This way, customers can fine-tune the language side of a Qwen3.5/3.6-VL model without paying to retrain weights they didn't want to touch.
Custom Triton kernels. If a model family hits a kernel-level bottleneck, we elect to write the kernel ourselves using Triton rather than waiting for upstream to catch up. An example where we did this is a fused RMSNorm-gated kernel for Qwen3.5/3.6's gated-delta-net layers, which combines RMS normalization, weight offset, and a SiLU gate into one forward and backward pass. We use Triton for fused LoRA kernels too, and a handful of model-specific paths where the off-the-shelf forward was leaving real performance on the table. Triton is effective for this kind of optimizations since the iteration loop is so fast.
MoE under the hood. While we only expose dense fine-tuning to customers today, internally the runtime supports two different MoE kernel backends (ScatterMoE and SonicMoE) for several MoE models. Getting this to work alongside tensor parallelism required a significant set of patches. MoE blocks use DTensor inputs that don’t compose cleanly with mixed Tensor and DTensor operations, so we wrote an adapter that does the conversion at the block boundary. We had to make weight layouts convertible so that checkpoints saved in one format could be loaded in vLLM. We haven’t released this to customers in production yet, but the runtime is ready for when we do.
Designing for multimodal and agentic from the start. A lot of training stacks add multimodal as an afterthought, we didn’t. From the beginning, we made several design decisions to make multimodal and agentic training seamless. We added large-scale image URL processing and custom chat-template logic so that we can handle multi-turn and tool call payloads across different model types.
And that’s just the tip of the iceberg. Adapter-only state-dict gathering so checkpointing a 30B LoRA doesn't OOM. Deferred tensor-parallel weight loading so model load doesn't generate NCCL collectives. A context-parallel aware evaluation mixin so evals don't deadlock at epoch boundaries. We keep the pattern consistent across all of these fixes. Staying HF Trainer/PyTorch native for the trainer loop benefits us because it lets us iterate fast and own the kernels/runtime patches.
## Mid training evals
Once a training job is running, we want to check that the model is actually learning the right thing. We decided to err away from validation losses alone. It predicts how accurately the model predicts the next token on held-out data, but does not translate to how well it actually performs on the assigned task. We’ve seen plenty of runs where the loss curve looks beautiful and the model is broken. The causes vary, from chat-template mismatches to tokenizer regressions on special tokens and sample packing bugs that corrupt position embeddings. The loss curve doesn’t catch any of these, because the loss is computed under the exact same conditions that produced the troublesome behavior.
Our approach is to run an eval on the actual task, mid-training, against the actual model. At set intervals during training, we spin up a vLLM inference container with the in-flight weights and generate responses against the held-out evaluation dataset, then run an LLM judge against each response with a rubric. In addition to that, we generate the rubric from a sample of the customer’s dataset. This way, the judge knows what the task looks like and what a good answer is. Every sample gets a numeric score and a written assessment.
This gives you a curve showing actual performance, allowing you to easily see when the model started improving, where it plateaued, and if it started overfitting.
This is also the correct way to carry out hyperparameter sweeps. Comparing loss curves between two configs tells you which one converged faster on next-token prediction. Comparing rubric scores tells you which one actually does the task better. Those aren't always the same answer, and when they disagree, the rubric is right.
We use these same scores to choose which checkpoint to deploy. We care about the one that did the best at the task. This also allows us to run longer runs to see when exactly we pass the line between learning and overfitting. Afterwards, we can deploy the best model with one click.
## One click to production
The last step is hosting. When a training run finishes, a final checkpoint is stored in a format that can be directly loaded by our inference engine. There’s no export step and no “where do I host this” step. The checkpoint is a Hugging Face directory with everything you need to run inference.
The trained model is automatically registered in your account as a deployable artifact. When you choose to deploy, our hosting worker pulls the checkpoint and then boots an inference server. Within a few minutes the model has a stable endpoint and is ready to serve requests.
Now you are back to where you started, collecting inferences on your new model. Build your next dataset from the outputs when you’re ready for a v2. This loop works because we own the whole process. You can observe your inferences, build a dataset, train and evaluate a model, then deploy it without ever leaving Catalyst.
## What is this for
We wanted to make it seamless to be able to switch to fine-tuned models, including all of training, evaluating, and deploying them.
If you build an application on top of a frontier model today, you relinquish all control. You are stuck with whatever that model charges with whatever latency the API delivers. The quality of response is how good that model happens to be at your task.
With Catalyst you don’t need to live with that. You route your traffic through our proxy and continue using the model you are using today. After you’ve collected some traffic, you are only a few clicks away from a new model. We build the dataset, configure the run, train the model, evaluate it against your task, pick the best checkpoint and deploy it. The result is a smaller, faster, cheaper model that does your task as well as the frontier model did. Often it does it better, because it's been trained on what you actually do instead of the entire internet.
While we’ve given lots of information about what goes on behind the scenes, there’s a lot more sauce in making all of this work end to end. We are also frequently monitoring any issues and deploying patches to make sure that this system improves over time. This originally started as a fine-tuning pipeline for one-shot tasks, but has grown to be able to train agentic models.
The biggest outcome from all of this has been the speed at which we’ve been able to ship trained models. Originally these used to be month-long engagements to collect the data, evaluate, get to train, and finally deploy. Now we are able to iterate and ship models sometimes as soon as the same day. It is also much cheaper for customers to consider training now.
If you want to learn more, check us out on inference.net or shoot me a DM!
## 相关链接
- [Amar Singh](https://x.com/AmarSVS)
- [@AmarSVS](https://x.com/AmarSVS)
- [1.3K](https://x.com/AmarSVS/status/2059067870743875684/analytics)
- [inference.net](https://inference.net/)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [8:23 AM · May 26, 2026](https://x.com/AmarSVS/status/2059067870743875684)
- [1,365 Views](https://x.com/AmarSVS/status/2059067870743875684/analytics)
---
*导出时间: 2026/5/26 11:45:02*
---
## 中文翻译
# 我们如何构建自动化的 LLM 训练基础设施
**作者**： Amar Singh
**日期**： 2026-05-26T00:23:30.000Z
**来源**： [https://x.com/AmarSVS/status/2059067870743875684](https://x.com/AmarSVS/status/2059067870743875684)
---
我们有一个愿景：打造一个完全集成的端到端 LLM 训练平台。我们希望让用户能够极其轻松地训练和部署随时间不断改进的小型模型。你将推理请求路由到我们的平台，构建数据集，然后训练并部署模型。此后，我们会持续收集你的推理数据，以便对模型进行后续改进。
这个理念很简单，但构建它的过程比我们预想的要难得多。
我想谈谈我们是怎样让“训练”按钮跑起来的。这涉及一大堆通常只存在于实验室和研究人员集体智慧与 bash 脚本中的问题：
- 数据集需要具备正确的格式，超参数必须与数据集相匹配。
- 你必须选择合适的模型和训练配置。
- 需要有可用的 GPU。
- 运行环境必须支持所选的模型家族，具备正确的内核、并行策略和检查点格式。
- 你的评估必须衡量真正重要的东西。
- 训练完成后，你训练出的模型应该离部署只差一次点击。
- 整个流程中可能出现的各种小众边缘情况。
所有这些障碍层层叠加，使整个流水线的构建变得困难。本文就是要讲我们如何应对并解决这一切。
## 数据进，数据集出
在训练任何东西之前，我们需要一个数据集。客户通过三种方式将数据接入 Catalyst：将他们的推理调用通过我们的代理路由，上传一个包含其在别处收集的推理数据的文件，或者通过我们兼容 OTEL 的 Catalyst Tracing SDK 发送智能体轨迹。
我们必须对这些样本进行规范化，使其成为我们流水线的有效输入。这意味着我们必须处理各种不同的格式问题，比如 Anthropic 将 base64 图片放在 source 对象里，OpenAI 使用 image_url，还有 Gemini 通过其 API 带来的混乱。我们在数据摄入阶段处理所有这些问题，以便后续顺利进入训练器。
之后，有一个数据预处理流水线负责清洗数据集。我们会丢弃空消息和过长的样本，并从训练集中剔除属于评估集的数据点。还有其他一些步骤，但其目的都是让我们对输入训练循环的内容有信心。
最近，我们还加入了 HALO 来处理更大规模的智能体数据集，因为轨迹分析起来尤其困难。我们会寻找可能被忽视的语义模式，比如模型拒绝回答或格式错误。在某些情况下，我们希望进行拒绝采样，以便为我们训练的模型设定更高的标准。
这些步骤大多是业界的常见做法，但其中许多都是在我们手动训练模型吸取教训之后才加入的。这条流水线是在过去一年中经过多次迭代逐步构建起来的。
## 我们替你选好超参数
如果你熟悉模型训练，就知道有一大堆超参数可以调整。这是最难做对的事情之一，尤其是因为配置错误会引发各种小众错误，不过我们有一套带有明确主张的系统来推算这些值。
当你点击“训练”时，我们会先对数据集启动一个分析步骤。我们会查看输入/输出字符统计、语义内容、模态等特征。基于这些，我们有一棵复杂的决策树，用来决定启用哪些参数，以便在速度和精度上同时优化训练运行。我们会调整样本打包、并行策略、学习率、epoch 长度、批量大小和梯度累积、视觉塔以及哪些参数被冻结等，还有更多。
除此之外，我们还有一个配方（recipe）层，也就是我们精心设计的起点。一个配方会给出基础模型、裁判模型、LoRA 拓扑、FSDP 版本、学习率和评估节奏。我们再叠加你数据集的统计数据，对配方进行调整，以确保设置合理。
运行评估时，我们会根据模型参数量和评估规模来确定沙箱的大小。更大的运行会分配更多 GPU，“更大”以模型大小或待处理样本数量来界定。我们还支持将评估路由到不同的 GPU 目标。每 GPU 内存利用率、最大并发序列数、生成批量大小和工作线程数会从一个按评估运行所在 GPU 类型索引的表中填写。我们会严格测试不同的硬件类型，以确保不会因此产生模型漂移。
如果我们认为某些参数需要微调，也可以覆盖并手动配置这些参数。
## 哪里有算力，就去哪里
训练任务需要 GPU，我们需要决定用哪些 GPU 以及从哪里获取。
我们使用自己的 GPU 集群作为主要训练资源池，因为这给了我们一层可预测性和控制力。当需要额外容量时，我们会回退到无服务器（serverless）GPU 提供商，否则客户会在高峰负载时经历漫长的排队等待。我们需要在两者之间取得平衡：虽然集群每 GPU 小时成本更低、更可靠，但我们不想在用量低谷时浪费容量。另一方面，如果始终在无服务器环境上训练，我们将承担更高的 GPU 成本和更高的错误率。我们构建了一个沿着优先级阶梯逐级检查的调度器，它会检查每个提供商的可用性，并在有容量时于数秒内分配任务。
由于我们是在多个提供商之间运行的，无法将运行时绑定到单一目标，因此我们的容器是与提供商无关的。它以命令行参数和载荷的形式接收任务配置，并通过签名的 webhook 进行汇报。训练容器中不包含任何编排逻辑。
一个有趣的结果来自无服务器提供商施加的时间限制。大多数提供商都会限制沙箱在被抢占之前可以运行多长时间。24 小时是常见的上限，而对大型模型进行严肃的微调则需要更长的时间。
我们的解决方案是通过检查点（checkpointing）进行滚动。运行时会定期将一个完全可恢复的检查点上传到持久存储。在沙箱即将被抢占前大约一小时，我们的控制平面会从最近的检查点启动一个新的训练任务，并停止旧任务。从客户的角度来看，这是一个连续的任务。而在基础设施方面，该任务在其生命周期内可能存在于许多不同的位置。
这使我们能够在没有任何沙箱能存活那么久的基础设施上运行多天的微调。它同时也兼作出问题时的恢复机制。如果发生运行时崩溃、NaN 损失、节点故障等，任务会从其最后一个可恢复的检查点继续进行。
## 训练运行时
几个月后，我们选定了训练运行时技术栈并坚持使用它。当时我们并没有意识到，这后来成为我们所做的最重要的决定之一。
我们从 Megatron-Bridge 和 Megatron-Core 生态系统起步。从纸面上看这是显而易见的：Megatron 拥有针对超大型模型最成熟的并行方案。其内核经过了实战检验，其渊源包括了该领域一些最具雄心的训练任务。我们让它跑了起来，并开始在上面运行任务。
然而，我们很快就开始碰壁。添加新模型是一场噩梦，而且训练速度本身相当慢，尤其是在 1000 亿参数以下的模型上。我们也无法从更大的内核生态中受益，因为大多数补丁都需要大规模的工程迁移。我们需要一个能够更快迭代和升级的方案。
为了解决这个问题，我们迁移到了基于 PyTorch 和 Hugging Face Trainer 的自定义框架。我们通过补丁的方式加入自己的并行实现和内核，以便能够从快速发展的前沿技术中受益。缺点是并行机制变得复杂得多，尤其是在长上下文或大参数模型的情况下。我们利用 Ring/Ulysses Attention、FSDP2 以及一些自定义 NCCL 操作，构建了自己的并行交互网络。
这也意味着我们在添加内核方面拥有了更细粒度的控制。我们将 ScatterMoE/SonicMoE 集成到了我们武器库中的 MoE 模型。我们能够将训练速度提升一个数量级，以至于我们的训练迭代步骤比某些 Megatron Core 循环本身还要快。
我们还需要考虑在这些并行设置下的自定义损失函数，例如在上下文被分片时计算损失值，或通过样本打包进行掩码处理。由于模型处理多模态掩码的方式各不相同，我们必须添加自定义的、针对特定模型的图像/视频通路，以便也能有效地训练它们。
其他一些注意事项：
FSDP2 下的冻结视觉塔预分片。当你微调视觉-语言模型时，通常希望保持视觉塔冻结。训练它的成本很高，而且下游任务通常是语言任务。FSDP2 原生并不支持某些子树被冻结、其他子树可训练的模块树，因为根包装会将它们组合在一起，导致梯度记账出错。我们通过在模型加载时检测冻结的视觉情况，并在递归包装到达之前先对视觉子树进行分片来解决这个问题。这样，客户可以微调 Qwen3.5/3.6-VL 模型的语言部分，而无需为重新训练他们不想触及的权重付费。
自定义 Triton 内核。如果某个模型家族遇到了内核层面的瓶颈，我们会选择用 Triton 自己编写内核，而不是等待上游赶上。我们这样做的一个例子是为 Qwen3.5/3.6 的 gated-delta-net 层编写的融合 RMSNorm 门控内核，它将 RMS 归一化、权重偏移和 SiLU 门控合并到一次前向和反向传播中。我们也将 Triton 用于融合 LoRA 内核，以及一些现成前向实现会浪费真实性能的模型特定路径。Triton 对这类优化非常有效，因为其迭代循环非常快。
底层实现中的 MoE。虽然我们目前只向客户开放稠密模型微调，但在内部，运行时为多个 MoE 模型支持两种不同的 MoE 内核后端（ScatterMoE 和 SonicMoE）。要让它与张量并行协同工作，需要进行大量的补丁开发。MoE 块使用的 DTensor 输入无法与混合 Tensor 和 DTensor 操作很好地组合，因此我们编写了一个在块边界进行转换的适配器。我们还必须使权重布局可转换，以便以一种格式保存的检查点能够在 vLLM 中加载。我们还没有在生产环境中向客户发布这一功能，但运行时已经为此做好了准备。
从一开始就为多模态和智能体而设计。许多训练技术栈将多模态作为事后的补充，而我们没有。从一开始，我们就做出了几项设计决策，以使多模态和智能体训练无缝衔接。我们添加了大规模的图像 URL 处理和自定义聊天模板逻辑，以便能够处理跨不同模型类型的多轮对话和工具调用载荷。
而这还只是冰山一角。仅适配器的状态字典收集，使得保存 30B LoRA 检查点时不会 OOM。延迟的张量并行权重加载，使模型加载不会产生 NCCL 集合通信。上下文并行感知的评估 mixin，避免评估在 epoch 边界处死锁。我们在所有这些修复中保持模式一致。在训练循环中坚持使用 HF Trainer/PyTorch 原生方案对我们很有利，因为它让我们能够快速迭代，并掌握内核和运行时补丁。
## 训练中途评估
一旦训练任务开始运行，我们就想检查模型是否真的在学习正确的东西。我们决定不单纯依赖验证损失。它预测的是模型在留出数据上预测下一个 token 的准确度，但并不能反映模型在指定任务上的实际表现。我们见过很多次运行，损失曲线非常漂亮，但模型却是坏的。原因各不相同，从聊天模板不匹配，到分词器对特殊 token 的回归，再到破坏位置嵌入的样本打包 bug。损失曲线无法捕获这些问题，因为损失恰恰是在产生这些问题的相同条件下计算出来的。
我们的做法是在训练中途对实际任务、实际模型进行评估。在训练过程中按固定间隔，我们启动一个带有当前权重的 vLLM 推理容器，针对留出的评估数据集生成回复，然后使用评分标准（rubric）通过 LLM 裁判对每个回复进行评判。此外，我们还从客户数据集的样本中生成评分标准。这样，裁判就知道任务长什么样、什么是好答案。每个样本都会得到一个数值分数和一段书面评语。
这会给你一条展示实际表现的曲线，让你可以轻松看出模型从什么时候开始改进、在哪里进入平台期，以及是否开始过拟合。
这也是进行超参数扫描的正确方式。比较两个配置之间的损失曲线，只能告诉你哪个在下一 token 预测上收敛更快；而比较评分标准分数，才能告诉你哪个在任务上实际做得更好。这两者并不总是同一个答案，而当它们不一致时，评分标准才是对的。
我们同样用这些分数来选择要部署哪个检查点。我们关心的是在任务上表现最好的那个。这也使我们能够进行更长时间的训练，以确切观察我们何时越过学习与过拟合之间的界线。之后，我们可以一键部署最佳模型。
## 一键进入生产环境
最后一步是托管。当训练运行结束时，最终检查点会以一种可直接被我们的推理引擎加载的格式存储。没有导出步骤，也没有“该把它托管在哪”的步骤。这个检查点就是一个 Hugging Face 目录，包含运行推理所需的一切。
训练好的模型会自动注册到你的账户中，成为一个可部署的产物。当你选择部署时，我们的托管工作进程会拉取检查点，然后启动一个推理服务器。几分钟内，模型就拥有一个稳定的端点，可以开始处理请求。
这样你就回到了起点——在新模型上收集推理数据。当你准备做 v2 时，再从这些输出构建下一个数据集。这个循环之所以有效，是因为我们掌控了整个流程。你可以观察推理结果、构建数据集、训练和评估模型，然后部署它——全程无需离开 Catalyst。
## 这是用来做什么的
我们希望让切换到微调模型变得无缝，涵盖训练、评估和部署的全部环节。
如果你今天在一个前沿模型之上构建应用，你就放弃了所有控制权。你只能接受该模型的收费和 API 的延迟。回复的质量取决于该模型在你的任务上碰巧表现如何。
有了 Catalyst，你不必忍受这些。你通过我们的代理路由流量，继续使用你今天正在使用的模型。在你收集了一些流量之后，只需点击几下，就能得到一个新模型。我们构建数据集、配置运行、训练模型、针对你的任务进行评估、挑选最佳检查点并部署。结果是一个更小、更快、更便宜的模型，在你的任务上做得和前沿模型一样好。通常它甚至做得更好，因为它是在你实际做的事情上训练的，而不是整个互联网。
虽然我们已经介绍了不少幕后发生的事情，但要让这一切端到端地运转起来，还有很多秘诀。我们也在持续监控问题并部署补丁，确保这个系统随时间不断改进。它最初只是一个面向单次任务（one-shot）的微调流水线，如今已成长为能够训练智能体（agentic）模型。
所有这一切最大的成果，是我们交付训练模型的速度。以前，从收集数据、评估、训练到最终部署，往往是一个月的合作周期。现在我们能够快速迭代和交付模型，有时当天就能完成。现在客户考虑训练的成本也低得多。
如果你想了解更多，请访问 inference.net，或者给我发私信！
## 相关链接
- [Amar Singh](https://x.com/AmarSVS)
- [@AmarSVS](https://x.com/AmarSVS)
- [1.3K](https://x.com/AmarSVS/status/2059067870743875684/analytics)
- [inference.net](https://inference.net/)
- [升级到 Premium](https://x.com/i/premium_sign_up)
- [2026年5月26日 上午8:23](https://x.com/AmarSVS/status/2059067870743875684)
- [1,365 次浏览](https://x.com/AmarSVS/status/2059067870743875684/analytics)
---
*导出时间: 2026/5/26 11:45:02*
