我们如何构建自动化的 LLM 训练基础设施

✍ Amar Singh🕐 2026-05-26📦 17.0 KB 🟢 已读 𝕏 文章列表

文章详细介绍了作者团队构建端到端 LLM 训练平台（Catalyst）的技术历程。通过解决数据标准化、超参数自动配置、多 GPU 提供商调度及无服务器环境下的长时训练恢复等挑战，实现了用户一键微调模型的愿景。

# How we built our automatic LLM training infrastructure **作者**: Amar Singh **日期**: 2026-05-26T00:23:30.000Z **来源**: [https://x.com/AmarSVS/status/2059067870743875684](https://x.com/AmarSVS/status/2059067870743875684) --- ![Image](/doc/2248/image/img_d55258cfbc83.jpg) We had a vision of a fully integrated end-to-end LLM training platform. We wanted to make it dead easy for a user to train and deploy small models that get better over time. You route your inferences through our platform, build a dataset, then train and deploy a model. Afterwards, we continue collecting your inferences to make follow-up improvements to the model. It’s a simple pitch, but building it was harder than we expected. I’m going to talk about how we got the “Train” button to work. This involved a whole host of issues that usually live in the collective mindshare and bash scripts of labs and researchers: - The dataset needs to be in the right shape and the hyperparameters must fit the dataset. - You must select the right model and training configuration. - GPUs need to be available. - The runtime must support the chosen model family, with the correct kernels, parallelism and checkpoint format. - Your evals have to measure something that matters. - When the training is done, your trained model needs to be one click from deployment. - A whole bunch of niche edge cases that might arise in the whole process. These are all barriers that compound to make the whole pipeline difficult to build. This post is meant to talk about how we tackled and solved all of this. ![Image](/doc/2248/image/img_69a07fc370ce.jpg) ## Data in, dataset out Before training anything, we need a dataset. Customers get data into Catalyst one of three ways: they route their inference calls through our proxy, they upload a file containing their inferences that were collected elsewhere, or they send agent traces through our OTEL-compatible Catalyst Tracing SDK. We have to normalize these samples such that they can be valid inputs to our pipeline. This means that we have to address different formatting issues like Anthropic putting base64 images inside a source object, OpenAI using image_url, and the mess that Gemini puts up through their API. We handle all of this at ingest so that we can continue on into the trainer. After that, there is a data preprocessing pipeline that is in charge of cleaning the dataset. We drop empty messages and excessively long samples, and remove evaluation set data points from the train set. There are a few more steps, but the point of this is to have confidence in what we feed into the training loops. Recently, we have also added HALO to handle larger agentic datasets, as traces can be particularly hard to analyze. We look for semantic patterns that might be overlooked like model refusals or formatting errors. In some cases, we want to do rejection sampling such that we can set a higher bar for the model that we train. These steps are mostly common practice, but many of these have been added after lessons from training models manually. This pipeline has been built over many iterations through the last year. ## We pick your hyperparameters for you If you are familiar with training models, you know that there is a large subset of hyperparameters that can be tweaked. This has been one of the hardest things to get right, especially because of all the niche errors that arise with misconfigurations, but we have an opinionated system to figure out these values. When you click “train”, we start an analysis step over the dataset. We look at inputs like input/output character statistics, semantic matter, modalities, and other features. Based on this, we have a complex decision tree of which parameters to enable to optimize the training run for both speed and accuracy. We adjust things like sample packing, parallelisms, learning rates, epoch lengths, batch sizes and gradient accumulations, vision towers and what parameters get frozen, and more. On top of that we have a recipe layer, our curated starting points. A recipe gives us the base model, the judge model, the LoRA topology, the FSDP version, the learning rate, and the eval cadence. We add your dataset’s statistics on top, tweaking the recipe to ensure we have appropriate settings. When running evals, we size the sandbox based on the number of model parameters and evaluation size. Larger runs get more GPUs, defining larger based on either the model size or the number of samples to process. We also support routing the evaluations to different GPU targets. Per-GPU memory utilization, max concurrent sequences, generation batch size, and worker thread counts get filled in from a table keyed on whatever GPU type the eval is running on. We rigorously test different hardware types to make sure that there is no model drift because of it. We can also override and manually configure these parameters if we ever think something should be tweaked. ![Image](/doc/2248/image/img_e23b0fec1921.jpg) ## Wherever there’s compute Training jobs need GPUs, we needed to decide which ones and where to get them. We use our own GPU cluster as our primary training pool, as that gives us a layer of predictability and control. When we need excess capacity we fall back to serverless GPU providers, as without this customers would experience long queue times during peak load. We needed to strike a balance here, as while the cluster is cheaper per GPU hour and more reliable, we didn’t want to waste capacity during usage troughs. On the other hand, if we always trained on serverless we would absorb higher GPU costs and higher error rates. We built a scheduler that walks a priority ladder, checking availability from each provider and assigning the job in seconds when capacity is there. Since we are running across multiple providers we couldn’t tie our runtime to a single target, and so our container is provider-agnostic. It takes a job config as CLI arguments and a payload, and it reports back over signed webhooks. No orchestration logic lives in the training container. An interesting consequence comes from the time limits that serverless providers impose. Most cap how long a sandbox can run before they are pre-empted. Twenty-four hours is common, and serious fine-tunes on a large model take longer than that. Our solution was rollover via checkpointing. Periodically, the runtime uploads a fully resumable checkpoint to durable storage. About an hour before the sandbox gets pre-empted, our control plane launches a new training job from the most recent checkpoint and stops the old one. From the customer’s perspective, there is one continuous job. On the infrastructure side, that job may have existed in many places over its lifetime. This allows us to run multi-day fine-tunes on infrastructure where no sandbox lives that long. It also doubles as a recovery mechanism for when something does go wrong. If there’s a runtime crash, NaN loss, node failure, etc., the job gets picked up from its last resumable checkpoint and keeps going. ## The training runtime A few months in we picked our training runtime stack and committed to it. We didn’t realize it at the time, but that turned out to be one of the most consequential decisions we made. We started with the Megatron-Bridge and Megatron-Core ecosystem. On paper it was obvious, Megatron had the most mature parallelism story for very large models. The kernels were battle tested, and the lineage includes some of the most ambitious training runs in the field. We got it working and started running jobs on it. We quickly started to run into hard walls, however. Adding new models was a nightmare and the training speeds themselves were pretty slow, especially on sub-100B models. We also could not benefit from the larger kernel ecosystem since most of these patches required large engineering migrations. We needed something where we could iterate and upgrade faster. To solve that, we moved to our custom framework on top of PyTorch and Hugging Face Trainer. We patch with our implementations of parallelism and kernels so we can benefit from the fast-moving frontier. The downside was that parallelisms got much more complex, especially with long context or large parameter models. We created our own network of parallelism interactions using Ring/Ulysses Attention, FSDP2, and some custom NCCL operations. This also meant that we had more fine grained control over adding kernels. We integrated ScatterMoE/SonicMoE to the MoE models that we had in our arsenal. We were able to speed up trainings by an order of magnitude, to the point where we had faster training iteration steps than some Megatron Core loops themselves. We also had to account for custom loss functions through these parallelism settings, like calculating the loss values when the context is sharded or masking through sample packing. With models having different ways to process multimodal masking, we had to add custom model-specific image/video pathways so that we could train with them effectively as well. Some other considerations: Frozen vision tower pre-sharding under FSDP2. When you fine-tune a vision-language model, you usually want to keep the vision tower frozen. It’s expensive to train and the downstream task is usually a language task anyway. FSDP2 doesn’t natively handle a module tree where some subtrees are frozen and others are trainable, as the root wrap groups them together and the gradient bookkeeping breaks. We solved this by detecting the frozen vision case during model loading and sharding the vision subtree first, before the recursive wrap reaches it. This way, customers can fine-tune the language side of a Qwen3.5/3.6-VL model without paying to retrain weights they didn't want to touch. Custom Triton kernels. If a model family hits a kernel-level bottleneck, we elect to write the kernel ourselves using Triton rather than waiting for upstream to catch up. An example where we did this is a fused RMSNorm-gated kernel for Qwen3.5/3.6's gated-delta-net layers, which combines RMS normalization, weight offset, and a SiLU gate into one forward and backward pass. We use Triton for fused LoRA kernels too, and a handful of model-specific paths where the off-the-shelf forward was leaving real performance on the table. Triton is effective for this kind of optimizations since the iteration loop is so fast. MoE under the hood. While we only expose dense fine-tuning to customers today, internally the runtime supports two different MoE kernel backends (ScatterMoE and SonicMoE) for several MoE models. Getting this to work alongside tensor parallelism required a significant set of patches. MoE blocks use DTensor inputs that don’t compose cleanly with mixed Tensor and DTensor operations, so we wrote an adapter that does the conversion at the block boundary. We had to make weight layouts convertible so that checkpoints saved in one format could be loaded in vLLM. We haven’t released this to customers in production yet, but the runtime is ready for when we do. Designing for multimodal and agentic from the start. A lot of training stacks add multimodal as an afterthought, we didn’t. From the beginning, we made several design decisions to make multimodal and agentic training seamless. We added large-scale image URL processing and custom chat-template logic so that we can handle multi-turn and tool call payloads across different model types. And that’s just the tip of the iceberg. Adapter-only state-dict gathering so checkpointing a 30B LoRA doesn't OOM. Deferred tensor-parallel weight loading so model load doesn't generate NCCL collectives. A context-parallel aware evaluation mixin so evals don't deadlock at epoch boundaries. We keep the pattern consistent across all of these fixes. Staying HF Trainer/PyTorch native for the trainer loop benefits us because it lets us iterate fast and own the kernels/runtime patches. ![Image](/doc/2248/image/img_be2700f5cb2d.jpg) ## Mid training evals Once a training job is running, we want to check that the model is actually learning the right thing. We decided to err away from validation losses alone. It predicts how accurately the model predicts the next token on held-out data, but does not translate to how well it actually performs on the assigned task. We’ve seen plenty of runs where the loss curve looks beautiful and the model is broken. The causes vary, from chat-template mismatches to tokenizer regressions on special tokens and sample packing bugs that corrupt position embeddings. The loss curve doesn’t catch any of these, because the loss is computed under the exact same conditions that produced the troublesome behavior. Our approach is to run an eval on the actual task, mid-training, against the actual model. At set intervals during training, we spin up a vLLM inference container with the in-flight weights and generate responses against the held-out evaluation dataset, then run an LLM judge against each response with a rubric. In addition to that, we generate the rubric from a sample of the customer’s dataset. This way, the judge knows what the task looks like and what a good answer is. Every sample gets a numeric score and a written assessment. This gives you a curve showing actual performance, allowing you to easily see when the model started improving, where it plateaued, and if it started overfitting. This is also the correct way to carry out hyperparameter sweeps. Comparing loss curves between two configs tells you which one converged faster on next-token prediction. Comparing rubric scores tells you which one actually does the task better. Those aren't always the same answer, and when they disagree, the rubric is right. We use these same scores to choose which checkpoint to deploy. We care about the one that did the best at the task. This also allows us to run longer runs to see when exactly we pass the line between learning and overfitting. Afterwards, we can deploy the best model with one click. ![Image](/doc/2248/image/img_269f158c85bf.jpg) ## One click to production The last step is hosting. When a training run finishes, a final checkpoint is stored in a format that can be directly loaded by our inference engine. There’s no export step and no “where do I host this” step. The checkpoint is a Hugging Face directory with everything you need to run inference. The trained model is automatically registered in your account as a deployable artifact. When you choose to deploy, our hosting worker pulls the checkpoint and then boots an inference server. Within a few minutes the model has a stable endpoint and is ready to serve requests. Now you are back to where you started, collecting inferences on your new model. Build your next dataset from the outputs when you’re ready for a v2. This loop works because we own the whole process. You can observe your inferences, build a dataset, train and evaluate a model, then deploy it without ever leaving Catalyst. ## What is this for We wanted to make it seamless to be able to switch to fine-tuned models, including all of training, evaluating, and deploying them. If you build an application on top of a frontier model today, you relinquish all control. You are stuck with whatever that model charges with whatever latency the API delivers. The quality of response is how good that model happens to be at your task. With Catalyst you don’t need to live with that. You route your traffic through our proxy and continue using the model you are using today. After you’ve collected some traffic, you are only a few clicks away from a new model. We build the dataset, configure the run, train the model, evaluate it against your task, pick the best checkpoint and deploy it. The result is a smaller, faster, cheaper model that does your task as well as the frontier model did. Often it does it better, because it's been trained on what you actually do instead of the entire internet. While we’ve given lots of information about what goes on behind the scenes, there’s a lot more sauce in making all of this work end to end. We are also frequently monitoring any issues and deploying patches to make sure that this system improves over time. This originally started as a fine-tuning pipeline for one-shot tasks, but has grown to be able to train agentic models. The biggest outcome from all of this has been the speed at which we’ve been able to ship trained models. Originally these used to be month-long engagements to collect the data, evaluate, get to train, and finally deploy. Now we are able to iterate and ship models sometimes as soon as the same day. It is also much cheaper for customers to consider training now. If you want to learn more, check us out on inference.net or shoot me a DM! ## 相关链接 - [Amar Singh](https://x.com/AmarSVS) - [@AmarSVS](https://x.com/AmarSVS) - [1.3K](https://x.com/AmarSVS/status/2059067870743875684/analytics) - [inference.net](https://inference.net/) - [Upgrade to Premium](https://x.com/i/premium_sign_up) - [8:23 AM · May 26, 2026](https://x.com/AmarSVS/status/2059067870743875684) - [1,365 Views](https://x.com/AmarSVS/status/2059067870743875684/analytics) --- *导出时间: 2026/5/26 11:45:02* --- ## 中文翻译 # 我们是如何构建自动化 LLM 训练基础设施的 **作者**: Amar Singh **日期**: 2026-05-26T00:23:30.000Z **来源**: [https://x.com/AmarSVS/status/2059067870743875684](https://x.com/AmarSVS/status/2059067870743875684) --- ![Image](/doc/2248/image/img_d55258cfbc83.jpg) 我们要构建一个完全集成的端到端 LLM 训练平台。我们希望让用户能够极其轻松地训练和部署那些会随着时间推移而变得更好的小模型。你只需将推理请求路由到我们的平台，构建数据集，然后训练并部署模型。之后，我们会持续收集你的推理数据，以对模型进行后续改进。这个愿景听起来很简单，但实际构建起来比我们预期的要难得多。我将谈谈我们是如何让“训练”按钮运作起来的。这涉及到了一系列通常只存在于实验室和研究人员的集体经验和 Bash 脚本中的问题： - 数据集需要具备正确的格式，超参数必须与数据集相匹配。 - 你必须选择正确的模型和训练配置。 - 必须有可用的 GPU。 - 运行时必须支持所选的模型系列，包含正确的内核、并行策略和检查点格式。 - 你的评估指标必须衡量真正重要的东西。 - 训练完成后，你训练好的模型必须只需点击一下即可部署。 - 整个过程中可能出现的一堆小众边缘情况。这些都是层层叠加的障碍，导致整个流程难以构建。这篇文章旨在探讨我们是如何解决所有这些问题的。 ![Image](/doc/2248/image/img_69a07fc370ce.jpg) ## 数据输入，数据集输出在训练任何东西之前，我们需要一个数据集。客户主要通过三种方式将数据导入 Catalyst：通过我们的代理路由推理调用，上传包含在别处收集的推理数据的文件，或者通过我们兼容 OTEL 的 Catalyst Tracing SDK 发送 Agent 追踪数据。我们必须将这些样本标准化，使其成为我们流水线的有效输入。这意味着我们必须处理不同的格式问题，例如 Anthropic 将 base64 图片放在源对象中，OpenAI 使用 image_url，以及 Gemini 通过其 API 制造的混乱局面。我们在数据摄入时处理所有这些问题，以便能够顺利进入训练器。之后，有一个数据预处理流水线负责清洗数据集。我们会丢弃空消息和过长的样本，并从训练集中移除评估集的数据点。还有更多步骤，但这样做的目的是为了确保我们对输入到训练循环中的数据有信心。最近，我们还添加了 HALO 来处理更大的 Agent 数据集，因为追踪数据可能特别难以分析。我们会寻找那些容易被忽视的语义模式，例如模型拒绝回答或格式错误。在某些情况下，我们希望进行拒绝采样，以便为我们训练的模型设定更高的标准。这些步骤大多属于常规做法，但许多措施是在手动训练模型吸取教训后才添加的。这条流水线在过去一年中经过了多次迭代构建而成。 ## 我们帮你挑选超参数如果你熟悉模型训练，你应该知道有大量的超参数可以调整。这是最难做对的事情之一，尤其是因为配置错误会引发各种小众错误，但我们有一套带有独到见解的系统来确定这些数值。当你点击“训练”时，我们会对数据集启动一个分析步骤。我们会查看输入/输出字符统计、语义内容、模态以及其他特征。基于此，我们有一个复杂的决策树来决定启用哪些参数，以优化训练运行的速度和准确性。我们会调整样本打包、并行策略、学习率、训练轮数长度、批大小和梯度累积、视觉塔以及哪些参数需要冻结等内容。除此之外，我们还有一个配方层，这是我们精心策划的起点。配方为我们提供了基础模型、评估模型、LoRA 拓扑结构、FSDP 版本、学习率和评估频率。我们会在此基础上叠加你数据集的统计数据，微调配方以确保设置得当。在运行评估时，我们会根据模型参数的数量和评估规模来调整沙箱大小。规模较大的运行会获得更多的 GPU，这里的“较大”是指模型尺寸或要处理的样本数量。我们还支持将评估路由到不同的 GPU 目标。每个 GPU 的内存利用率、最大并发序列数、生成批大小和工作线程计数都会根据评估所运行的 GPU 类型从表中填充。我们会严格测试不同的硬件类型，以确保不会因此导致模型漂移。如果我们认为某些设置需要调整，也可以覆盖并手动配置这些参数。 ![Image](/doc/2248/image/img_e23b0fec1921.jpg) ## 有算力的地方就是训练场训练作业需要 GPU，我们需要决定使用哪种 GPU 以及从哪里获取它们。我们使用自己的 GPU 集群作为主要的训练池，因为这为我们提供了一层可预测性和控制力。当我们需要超额容量时，我们会回退到无服务器 GPU 提供商，因为没有这一点，客户在负载高峰期将面临很长的排队时间。我们需要在这里取得平衡：虽然集群每 GPU 小时的成本更低且更可靠，但我们不想在低谷期浪费容量。另一方面，如果我们总是在无服务器上训练，我们将承担更高的 GPU 成本和更高的错误率。我们构建了一个调度器，它会沿着优先级阶梯爬升，检查每个提供商的可用性，并在有容量时在几秒钟内分配作业。由于我们要跨多个提供商运行，我们不能将运行时绑定到单一目标上，因此我们的容器是与提供商无关的。它将作业配置作为 CLI 参数和有效载荷接收，并通过签名的 Webhook 报告状态。训练容器中不存在任何编排逻辑。无服务器提供商施加的时间限制带来了一个有趣的后果。大多数提供商限制了沙箱在被抢占前能运行的时间。24 小时是常见的上限，而在大型模型上进行认真的微调所需时间更长。我们的解决方案是通过检查点进行滚动交接。运行时会定期将完全可恢复的检查点上传到持久存储。在沙箱被抢占前大约一小时，我们的控制平面会从最新的检查点启动一个新的训练作业，并停止旧的作业。从客户的角度来看，这是一个连续的作业。在基础设施层面，该作业在其生命周期中可能存在于许多不同的地方。这使我们能够在没有任何沙箱能存活那么久的基础设施上运行持续数天的微调。它同时也充当了恢复机制，以防出现问题。如果发生运行时崩溃、NaN 损失、节点故障等，作业会从其最后一个可恢复的检查点恢复并继续运行。 ## 训练运行时几个月后，我们选择了我们的训练运行时技术栈并致力于此。当时我们没有意识到，但这结果是我们做出的最重大的决定之一。我们从 Megatron-Bridge 和 Megatron-Core 生态系统开始。理论上这很明显，Megatron 对于超大模型拥有最成熟的并行处理方案。其内核久经考验，其渊源包括该领域一些最雄心勃勃的训练运行。我们让它运转起来并开始在其上运行作业。然而，我们很快遇到了硬瓶颈。添加新模型是一场噩梦，训练速度本身也很慢，特别是在 100B 参数以下的模型上。我们也无法受益于更大的内核生态系统，因为大多数这些补丁需要大规模的工程迁移。我们需要一种能够让我们更快迭代和升级的东西。为了解决这个问题，我们转移到了 PyTorch 和 Hugging Face Trainer 之上的自定义框架。我们用自己的并行和内核实现进行打补丁，以便我们能受益于快速发展的前沿技术。缺点是并行变得更加复杂，特别是对于长上下文或大参数模型。我们使用 Ring/Ulysses Attention、FSDP2 和一些自定义 NCCL 操作创建了自己的并行交互网络。这也意味着我们对添加内核有了更细粒度的控制。我们将 ScatterMoE/SonicMoE 集成到了我们武器库中的 MoE 模型中。我们能够将训练速度提高了一个数量级，以至于我们的训练迭代步骤比某些 Megatron Core 循环本身还要快。我们还必须通过这些并行设置来考虑自定义损失函数，例如在上下文分片时计算损失值或通过样本打包进行掩码。由于模型处理多模态掩码的方式不同，我们必须添加自定义的特定于模型的图像/视频路径，以便我们也能有效地训练它们。其他一些考虑事项：在 FSDP2 下进行冻结视觉塔的预分片。当你微调视觉语言模型时，通常希望保持视觉塔冻结。训练它的成本很高，而且下游任务通常是一个语言任务。FSDP2 本身不处理某些子树冻结而另一些子树可训练的模块树，因为根包装将它们组合在一起，导致梯度记账中断。我们通过在模型加载期间检测冻结视觉情况并在递归包装到达之前先对视觉子树进行分片来解决这个问题。这样，客户可以微调 Qwen3.5/3.6-VL 模型的语言侧，而无需为重新训练他们不想触碰的权重买单。自定义 Triton 内核。如果某个模型系列遇到内核级别的瓶颈，我们选择使用 Triton 自己编写内核，而不是等待上游跟上。我们这样做的一个例子是 Qwen3.5/3.6 的 gated-delta-net 层的融合 RMSNorm 门控内核，它将 RMS 归一化、权重偏移和 SiLU 门控组合到一个前向和后向传递中。我们也对融合 LoRA 内核使用 Triton，以及一小部分现成前向传播未能发挥真实性能的特定于模型的路径。Triton 对这种优化非常有效，因为迭代循环非常快。底层支持 MoE。虽然我们目前只向客户公开密集微调，但在内部运行时支持几种 MoE 模型的两种不同 MoE 内核后端。为了使其与张量并行一起工作，需要大量的补丁。MoE 块使用 DTensor 输入，不能与混合张量和 DTensor 操作干净地组合，因此我们编写了一个适配器，在块边界进行转换。我们必须使权重布局可转换，以便以一种格式保存的检查点可以在 vLLM 中加载。我们尚未在生产环境中向客户发布此功能，但运行时已准备就绪。从一开始就为多模态和 Agent 设计。许多训练栈将多模态作为事后补充，而我们没有。从一开始，我们就做出了几个设计决策，使多模态和 Agent 训练无缝衔接。我们添加了大规模图像 URL 处理和自定义聊天模板逻辑，以便我们可以处理跨不同模型类型的多轮对话和工具调用有效载荷。这还只是冰山一角。仅适配器状态字典收集，以便检查点 30B 的 LoRA 不会导致内存溢出。延迟张量并行权重加载，以便模型加载不会产生 NCCL 集合操作。上下文并行感知评估混合，以便评估不会在轮次边界死锁。我们在所有这些修复中保持模式一致。保持训练器循环的原生 HF Trainer/PyTorch 性质对我们有利，因为它让我们能够快速迭代并拥有内核/运行时补丁。 ![Image](/doc/2248/image/img_be2700f5cb2d.jpg) ## 训练中期评估一旦训练作业开始运行，我们要检查模型确实在学习正确的东西。我们决定不仅仅依赖验证损失。它预测模型在保留数据上预测下一个 token 的准确程度，但这并不转化为其在分配的任务上的实际表现如何。我们见过很多运行，其损失曲线看起来很完美，但模型却是坏的。原因各异，从聊天模板不匹配到特殊 token 的分词器退化，以及破坏位置嵌入的样本打包错误。损失曲线无法捕捉其中任何一个，因为损失是在产生麻烦行为的完全相同条件下计算的。我们的方法是在实际任务上、在训练中期、针对实际模型运行评估。在训练期间的设定间隔，我们会启动一个带有正在运行权重的 vLLM 推理容器，根据保留的评估数据集生成响应，然后使用评分标准对每个响应运行 LLM 评判者。除此之外，我们还从客户数据集的样本中生成评分标准。这样，评判者就知道任务是什么样的，以及什么是好的答案。每个样本都会得到一个数字分数和书面评估。这会为你提供一条显示实际性能的曲线，让你可以轻松地看到模型何时开始改进，何时进入平台期，以及是否开始过拟合。这也是进行超参数扫描的正确方法。比较两个配置之间的损失曲线告诉你哪一个在下一个 token 预测上收敛得更快。比较评分标准分数告诉你哪一个实际上更好地完成任务。这些答案并不总是一致的，而当它们不一致时，评分标准是对的。我们使用这些相同的分数来选择要部署的检查点。我们关心的是在任务上表现最好的那个。这也允许我们运行更长时间的训练，以确切地看到我们何时跨越了学习与过拟合之间的界限。之后，我们可以一键部署最好的模型。 ![Image](/doc/2248/image/img_269f158c85bf.jpg) ## 一键投入生产最后一步是托管。当训练运行结束时，最终的检查点会以可以直接被我们的推理引擎加载的格式存储。没有导出步骤，也没有“我在哪里托管这个”的步骤。检查点是一个包含运行推理所需一切的 Hugging Face 目录。训练好的模型会自动在你的账户中注册为可部署的工件。当你选择部署时，我们的托管工作程序会拉取检查点，然后启动推理服务器。在几分钟内，模型就拥有了一个稳定的端点并准备好服务请求。现在你回到了起点，正在收集新模型上的推理数据。当你准备好 v2 版本时，可以从输出中构建下一个数据集。这个循环行之有效，因为我们拥有整个过程。你可以观察你的推理，构建数据集，训练和评估模型，然后部署它，而无需离开 Catalyst。 ## 这一切是为了什么我们要让切换到微调模型变得无缝衔接，包括所有的训练、评估和部署。如果你今天在某个前沿模型之上构建应用程序，你就放弃了所有的控制权。你被迫接受该模型收取的任何费用以及 API 交付的任何延迟。响应的质量取决于该模型在你的任务上恰好表现得多好。有了 Catalyst，你无需妥协。你将流量路由通过我们的代理，并继续使用你今天正在使用的模型。在你收集了一些流量后，你只需点击几下就能得到一个新模型。我们构建数据集，配置运行，训练模型，根据你的任务进行评估，挑选最好的检查点并部署它。结果是一个更小、更快、更便宜的模型，它在你的任务上表现得和前沿模型一样好。

🔗 相关文档