The Cutoff Is an Option: Single-Pass Multi-Label Classification with Adaptive Thresholding

Decision models answer one thing per question. We make one question return every label that applies by adding a single option, the cutoff, and training each label to beat it or fall below it. The cutoff is scored in the same pass and the same softmax as the labels, so the model sets a new bar for every input. It needs no new head and no change to the decision-model API. Without any tuning, it beats the best fixed threshold in blind review.

Highlights

• The cutoff is an option. One extra option, scored in the same softmax as the labels, sets a new threshold for every input. A label is kept when it outscores the cutoff.
• Multi-label in one pass, for LLM decision models. Today's LLM-based decision APIs (Jev, Microsoft-Decision-1, OpenAI's Decisions API, Cloudflare Clef, Decision 3.0 and others) return one answer per question. With the cutoff, one question returns every label that applies, from zero to many.
• No tuning needed. In blind review, the untuned cutoff (F1 0.850) beats the best fixed probability threshold, even one tuned on the test set itself: +0.028, 95% CI +0.012 to +0.044.
• Robust to list length. A fixed threshold's best value more than triples as option lists grow from under 50 to over 60. The cutoff's best setting barely moves.
• Calibrated. A label scored above 0.9 is right 98% of the time, with a calibration error of 0.037.
• Accurate and cheap. On a production tagging task it matches a frontier LLM at 1/48 to 1/8 of its cost per article. It beats commercial decision APIs asked once per label (F1 0.73 against 0.51–0.61), and replaces 54 forward passes with 1.
• Standard serving. It runs on an unmodified decision-model endpoint: 23 articles per second on one RTX 5090 in FP8.

1. The gap: one answer per question

A decision model reads an input and a typed question and returns probabilities instead of text. It generates nothing, so it never returns malformed output. Every decision API today offers the same three question types, and each returns one answer:

Listing every option in criteria does not change this. The probabilities returned with a choice or score answer form one distribution that sums to 1, and nothing in it says how many options apply.

Many real tasks are multi-label: each input belongs to any number of classes, including none. A news article can be about earnings, telecommunications and satellite services at once. A support ticket can need billing and account access at once. In our reference set, articles carry between 2 and 13 labels.

There are two ways to do multi-label with decision models today, and each has a cost:

• LLM-based decision models ask one yes/no question per label. That is how benchmarks of Jev run multi-label tasks, and how users of Microsoft-Decision-1, OpenAI's Decisions API and Cloudflare Clef do it.
• Cost: questions can share one HTTP request, but they are evaluated "in parallel and in isolation", and the open Decision 3.0 serving recipe runs one forward pass per question. Fifty labels mean fifty passes over the same input.
• Quality: each label is judged without seeing the others. An independent benchmark of Jev found that per-label yes/no answers over-predict "yes" (mean 0.21 against a true rate of 0.04), and needed thresholds tuned label by label.
• Encoder decision models add a multi-label head with a fixed threshold. Fastino's GLiNER2.5-Decide (a 340M DeBERTa encoder) returns "every label above the threshold". Vela 2.0 (a ModernBERT encoder) adds a Set question type.
• What's good: they are fast and run in one pass.
• What's fixed: the threshold is a constant tuned once, the same for every input. These models are small, with a 512-token context for GLiNER2.5-Decide, and score far below LLM decision models on hard decisions.

No LLM-based decision model offers multi-label in one question. That is the gap this paper closes, with a mechanism that also replaces the fixed threshold.

2. The method: the cutoff is an option

We ask one choice question that lists every candidate label, plus one extra option:

REF: "Cutoff: no remaining option is materially relevant"

The cutoff is part of the input. The client writes it into criteria like any other option, with the same fixed text on every request. The model reads it and returns a probability for it, as it does for each label.

Scoring

In one forward pass, the model reads the input and every option, and gives each option i a raw score zᵢ. That is the dot product of the option's readout row with the final hidden state, as in any LLM decision model. The cutoff is just another option, so it gets a score too: z_REF. The decision endpoint returns the usual softmax:

p_i = \frac{e^{z_i}}{\sum_j e^{z_j}}

The raw scores are not returned, but their differences survive the softmax exactly:

\log \frac{p_i}{p_{\mathrm{REF}}} = z_i - z_{\mathrm{REF}}

The decision rule

Keep label i when it outscores the cutoff:

z_i > z_{\mathrm{REF}} \iff p_i > p_{\mathrm{REF}}

So z_REF is the threshold, and the model computes it from this input, in the same pass as the labels. If nothing beats the cutoff, the answer is empty: "none of these" needs no extra machinery.

Training

We fine-tune on proprietary data with a binary loss on each label's margin over the cutoff:

\mathcal{L} = \sum_{i \ne \mathrm{REF}} \mathrm{BCE}\big(\sigma(z_i - z_{\mathrm{REF}}),\; y_i\big)

Here yᵢ is 1 when label i applies. The loss pushes relevant labels above the cutoff and irrelevant ones below it. Across many inputs, the model learns where to place the cutoff for each one:

• Few labels apply: the cutoff rises above everything else.
• Many labels apply: the cutoff sits below all of them.

Option order is shuffled at every step, so the position-based readout learns no position bias. The rest is a standard LoRA fine-tune of a 4B decision model.

The request

The request is an ordinary choice question, on the ordinary endpoint:

{
  "state": "A satellite communications company reports quarterly results; enterprise demand and equipment sales lift its outlook.",
  "questions": {
    "tags": {
      "type": "choice",
      "instructions": "Tag this story. Score every materially relevant option above REF and every other option below it.",
      "criteria": {
        "iab:A1": "IAB | Business and Finance | Industries > Telecommunications Industry",
        "iptc:B7": "IPTC | corporate earnings | Announcements and discussions of corporate earnings",
        "naics:C3": "NAICS | Information | Telecommunications > Satellite Telecommunications",
        "...": "one entry per candidate label",
        "REF": "Cutoff: no remaining option is materially relevant to the story"
      }
    }
  }
}

The client keeps every option whose probability exceeds REF's. The cutoff can also be shifted globally, keeping options with zᵢ − z_REF > c. The default c = 0 is what the loss trains for, and it scored best in blind review.

3. Why the cutoff belongs inside the softmax

Scoring the threshold as an option, rather than fixing it as a constant, gives four properties at once.

1. The bar moves per input. z_REF comes from the same forward pass, after the model has read the input and every candidate. A clear story pulls it down, so every relevant label clears it. A vague or off-topic input pushes it up, above labels that only look related.

2. It is invariant to list length. A softmax probability depends on how many options share it: 0.05 is a strong signal among 70 options and a weak one among 3. So a fixed probability threshold means different things on different inputs. The ratio pᵢ / p_REF cancels the shared denominator, so the rule reads the same with 20 options or 200.

3. "None of these" is free. When the cutoff outscores every label, the answer is empty. A fixed threshold either tags weak matches on junk input, or is raised until it loses recall on everything else.

4. Labels compete. Every label is scored while the model sees all the others and the cutoff, so a specific code can win over its broad parent. Per-label yes/no questions never see the alternatives.

Encoder models such as GLiNER2.5-Decide also read the text and all labels together, so property 4 is not unique to us. Properties 1 to 3 are new to decision models. They come from scoring the threshold itself, in the same distribution as the labels. The idea of a learned threshold class comes from relation extraction (ATLOP, section 10).

4. Evidence: the threshold mechanism alone

The cleanest test holds the model fixed and changes only how its output is read. We took one fine-tuned model's probabilities for every option, cutoff included, and turned them into tags six ways.

• Against blind reviewers, the untuned cutoff is best.
• It scores F1 0.850 with no tuning at all.
• The best fixed probability threshold, even tuned on the test set itself, scores 0.822. The difference is +0.028, with a paired-bootstrap 95% CI of +0.012 to +0.044.
• Tuning an offset on the cutoff does not improve it, because 0 is already the best setting.
• Against the frontier-model reference, which tags more exhaustively, both readings need tuning.
• A fixed threshold tuned on that reference edges out the untuned cutoff, by 0.019 (95% CI 0.007 to 0.031).
• With both tuned the same way, the cutoff wins: 0.763 against 0.751.
• Single answers lose badly. One choice per input finds only one label per article (F1 0.25–0.33). One per taxonomy and top-k cannot follow articles that need 2 labels in one taxonomy and none in another.

A fixed threshold has no safe default, and drifts with list length

A fixed probability threshold only works at a value found by tuning, and that value depends on how many options share the softmax.

The best fixed threshold more than triples between short and long option lists, because each extra option thins every probability. The cutoff's best setting barely moves: the ratio to REF cancels the list length, as section 3 predicts.

5. Evidence: the bar moves

If the cutoff were a disguised constant, it would take about the same share of probability on every input. It does not: across 401 articles it takes anywhere from 0.03% to 33% of the probability, a 1,000-fold range.

• A clear story pulls the bar down. A sports trade story puts 0.03% on the cutoff, and 6 labels clear it.
• A thin input pushes it up. A one-line regulatory filing heading puts 8.9% on the cutoff, and only 2 labels clear it.
• An off-topic input raises it above everything. A non-English legal-dispute headline, with no fitting industry or topic among its candidates, puts 33% on the cutoff, and nothing is kept.

A fixed probability threshold has no way to tell these inputs apart. It applies the same number to all three.

6. Evidence: the scores are calibrated

Each label is trained as a yes/no decision against the cutoff. So the sigmoid of its margin, σ(zᵢ − z_REF), should read as the probability that the label applies. We checked this on every candidate label in the reference set.

• High scores can be trusted. Labels scored above 0.9 are right 98% of the time. Labels scored below 0.1 are right 2% of the time, and they are most of the candidates.
• The middle is cautious. Between 0.1 and 0.5 the model under-claims. Those labels are dropped, but 32% to 55% of them are right. A negative offset c recovers them when a task wants exhaustive tags. In section 4, an offset of −1 (keep labels scored above 0.27) lifts F1 against the frontier reference from 0.732 to 0.763.
• One score, one meaning. Because each score is a probability against the cutoff, a team can route by it. For example: accept above 0.9, send 0.3 to 0.7 to review, drop the rest. A fixed softmax probability gives no such reading, because its meaning shifts with list length (section 4).

Calibration here is a side effect of the loss, not a separate step. We applied no temperature scaling or post-hoc fit.

7. Against today's decision models

7.1 The landscape

Decision models today fall into two families. Neither scores the threshold as part of the answer.

The cutoff gives LLM decision models what encoder models already have, one pass and labels that compete. It also adds what neither family has: a threshold that the model sets for each input.

7.2 Choice, score and ours on one article

The three standard question types answer different questions. None of them answers "which of these apply?". The table runs each approach on one article: a satellite communications company's quarterly results, with enterprise demand and equipment sales. The reference has 5 labels.

• choice caps the answer. It must return exactly one code per question. Five labels across 4 taxonomies means at least one taxonomy needs 2 codes, so at least one is always missed.
• score is the wrong shape. It returns a weighted level on one scale, for example 2.3 on 0 to 3. It cannot name which labels apply.
• Per-label noul over-tags. Each label is judged alone, and 20 of them say yes. That finds all 5 reference labels, but adds 15 more. The independent benchmark of Jev found the same bias on other tasks.
• The cutoff keeps 6. Five of them match the reference.

7.3 Many questions in one request are still many passes

Decision APIs let one request carry many questions. That saves round trips, not compute. The APIs evaluate the questions "in parallel and in isolation", and each question is its own forward pass over the same input. No question sees another's answer.

On our task, asking one question per label means 53.9 passes per article on average, against our 1. It processes 5.7 times the tokens: 11,236 per article against 1,985. The commercial APIs billed 5,229 to 10,810 tokens per article.

7.4 Accuracy and cost

• It matches the frontier LLM. F1 0.732 against 0.726 for GPT-6 Luna (low reasoning), on the frontier-model reference.
• It beats the decision APIs by 13 to 22 points. With one question per label, Cloudflare Clef scores 0.61, Microsoft-Decision-1 0.52, GPT-6 Luna Decisions 0.52 and Jev 1.14 0.51. Each threshold was tuned for best F1.
• Blind reviewers agree. In a blind five-way review of 160 articles, scored 1 to 5, the averages were 4.86 and 4.62 for two other frontier models, 4.66 for ours, 4.30 for GPT-6 Luna, the frontier LLM we replaced, and 3.16 for Qwen 3.7 Flash. In a separate pairwise review of 250 articles, reviewers preferred ours over that frontier LLM 117 to 41, with the rest tied.
• Why the decision APIs cost about as much as a frontier LLM, or more. A decision API is cheap per question. But multi-label use needs one question per candidate label, about 54 per article, and the APIs bill every question. An article came to 5,229 to 10,810 billed tokens. GPT-6 Luna reads the article and all candidates once and writes one short list. So the low price per question is paid 54 times. Microsoft-Decision-1 and Jev 1.14 end up at about 0.8 times GPT-6 Luna's cost per article, Clef at 1.4 times and GPT-6 Luna Decisions at 3.8 times. Our model asks one question per article, so it pays once.
• The cost gap is large at any load. On a fully used GPU, an article costs 1/48 of what the frontier LLM charges. On a rented GPU that sits mostly idle, it still costs 1/8.

8. Serving it

The cutoff needs nothing from the server that a decision model does not already have.

• No new endpoint. The request in section 2 is a standard choice question on a standard decision-model server (vLLM). The client reads the returned probabilities and applies the rule. There is no custom head and no server patch.
• One pass, no output tokens. Each article is about 2,000 tokens in and nothing out. A request cannot return malformed output or run long.
• Throughput. One RTX 5090 serves 23 articles per second in FP8.
• FP8 is safe. FP8 computed at load time and a pre-quantized FP8 checkpoint gave the same accuracy: F1 0.800 and 0.799 on our release set. On the labels where the two disagreed, neither was right significantly more often (77 against 63, p = 0.27).
• FP4 is not. NVFP4 changed the tag set on 51% of articles. F1 fell from 0.799 to 0.790, and recall on the finest taxonomy, NAICS, fell from 0.610 to 0.576. On the labels where FP8 and FP4 disagreed, FP8 was right 735 times and FP4 604 times (McNemar p < 0.001).

The FP4 result follows from the method. A multi-label answer is decided at the cutoff, and many labels sit close to it. Small numeric noise moves them across. A single-answer decision only changes when the top two options swap.

The same applies to batching. Batched serving shifts margins slightly with batch composition, so a label within a hair of the cutoff can flip between runs. Release checks should therefore count flips on labels with a clear margin, and set an F1 floor, rather than demand identical tag sets.

9. Limits and open questions

• No head-to-head with encoder models. We did not run GLiNER2.5-Decide or Vela 2.0 on this task. Our comparison with them is about mechanism, not measured accuracy.
• One task, one domain. All results come from tagging news articles against four industry and topic taxonomies. Other multi-label tasks may behave differently.
• Option cap. The readout scores up to 255 options, so one question holds up to 254 labels plus the cutoff. Larger label spaces need a retrieval step first. Ours retrieves about 55 candidates per article.
• One cutoff per question. We used one cutoff across all four taxonomies. A cutoff per group of options is a natural extension that we did not test.
• Fine-grained recall. Recall is lowest on the most fine-grained taxonomy, NAICS (0.61).
• Model-made references. The reference labels and the blind reviewers are LLMs. We did not measure agreement with human reviewers.
• Near-cutoff noise. Labels close to the cutoff can flip with batch shape or lower precision (section 8).