跳到主要内容
版本:最新版(未发布)

分类请求

分类器给每个请求贴个标签:它的主题、要不要事实核查、用户对上一条答案满不满意、或者他们是不是在要图。路由就按这些标签匹配。

信号模型标签
domainVela 1.0 Domain14 个领域:biology、business、chemistry、computer science、economics、engineering、health、history、law、math、other、philosophy、physics、psychology
fact_checkVela 1.0 FactCheckFACT_CHECK_NEEDED、NO_FACT_CHECK_NEEDED
user_feedbackVela 1.0 Feedbacksatisfied、need clarification、wrong answer、want different、no feedback
modalityVela 1.0 ModalityAR(文本)、DIFFUSION(图像)、BOTH
classifier你自己的模型你的标签

打开它​

把信号加上,在路由里用起来。模型由 router 在 CPU 上替你跑,不用别的:

routing:
signals:
domains:
- name: math
description: Mathematics and quantitative reasoning.
mmlu_categories: [math]
- name: computer science
description: Programming and computer science.
mmlu_categories: [computer science]
decisions:
- name: math-route
priority: 100
rules:
operator: AND
conditions:
- type: domain
name: math
modelRefs:
- model: math-model

选它跑在哪​

想自己挑设备、输入上限或换个模型,就写个部署,把功能绑上去。绑定名叫 domain_classifier、fact_check_classifier、feedback_detector 和 modality_detector,它们读的都是标签概率(label_distribution.v1):

global:
model_catalog:
deployments:
vela-domain:
provider: model_runtime
artifact: vllm-sr/Vela-1.0-Encoder-307M-Domain
device: cpu
input:
max_tokens: 2048
overflow: truncate
bindings:
domain_classifier:
deployment: vela-domain
contract: label_distribution.v1

overflow: truncate 拿长请求的前 2,048 个 token 分类,并如实报告截断了;reject(默认)则让超限输入的信号保持未知。

验一下​

直接问模型。单独起一个,或用任意服务它的 runtime:

这些 worker 级示例在含有 vllm-srun 的环境里运行(例如 Router 镜像)。Classify、embeddings、rerank 和 bundle 是 worker API;实例前端发布的是 System One 和 decision 请求。

vllm-srun serve vllm-sr/Vela-1.0-Encoder-307M-Domain --device cpu --port 8100
curl -s localhost:8100/v1/classify -H 'content-type: application/json' \
-d '{"input": ["What is the derivative of x squared?", "Fix this segfault in my C code."]}'

每条结果带最可能的 label 和按 labels 顺序排的全部 probabilities。经过 router 的话,发请求时带 x-vsr-debug: true 头:x-vsr-matched-domains 列出匹配上的领域,x-vsr-selected-decision 点名走了哪条路由。

用自己的分类器​

Hugging Face 上的 ModernBERT 或 mmBERT 序列分类器,可以直接当 classifier 信号用,标签随你定。钉住它的 revision、绑到规则上、按它的标签路由:

routing:
signals:
classifiers:
- name: ticket_topic
type: local
labels: [billing, shipping, other]
decisions:
- name: billing-desk
priority: 100
rules:
operator: AND
conditions:
- type: classifier
name: ticket_topic
label: billing
predicate:
gte: 0.5
modelRefs:
- model: support-model
model_bindings:
classifier.ticket_topic:
deployment: ticket-topics
contract: label_distribution.v1
global:
model_catalog:
deployments:
ticket-topics:
provider: model_runtime
artifact: your-org/ticket-topic-classifier
revision: 0123456789abcdef0123456789abcdef01234567
device: cpu
input:
max_tokens: 512
overflow: reject

labels 要按模型自己的标签顺序列(它的 id2label)。router 从 runtime 读模型的标签,和规则对不上就拒绝启动。互相独立的标签、又有公开工作点的模型,用 label_scores.v1;见分类器信号。