Lumera News AI News and analysis on artificial intelligence.

Models

Apertus 1.5 release notes detail multimodal features and benchmark trade-offs

October’s Apertus 1.5 release notes document models available since July, with multimodal inputs, expanded context and more evaluation detail—but also benchmark regressions and serving constraints.

The Swiss AI team published Apertus 1.5 release notes on October 15, 2026, detailing multimodal capabilities, training changes and evaluations for model weights it says were released on July 24. The October publication is a technical disclosure about an existing release—not a new model launch. 1

Apertus is developed by researchers at ETH Zurich, EPFL and the Swiss National Supercomputing Centre. Hugging Face hosts the release notes but is not the model’s developer. 2

The documentation gives users more material to inspect, including a versioned training-data catalog and structured evaluation records. Those artifacts should not be confused with an independent data audit or replication of every reported benchmark result. 3 4

What Apertus 1.5 documents

According to the official model documentation, Apertus 1.5 comes in 8-billion- and 70-billion-parameter versions continued from Apertus 1.0. The documented specifications include: 5

  • Text, image and audio inputs, with text-only output.
  • A 262,144-token context window, four times the previous release’s window.
  • Optional thinking mode, disabled by default.
  • Apache 2.0-licensed weights. 5

The team reports an additional 4 trillion pretraining tokens for the 8B model and 2 trillion for the 70B model, bringing cumulative totals to 19 trillion and 17 trillion respectively. Its release notes describe an early-fusion approach in which image and audio tokens enter the text sequence; audio understanding remains experimental. 5 1

The variants also differ in final alignment. The team says the released 8B received an additional charter-alignment SDPO stage, while the 70B stopped at DPO. The technical report attributes an initially observed 70B SDPO regression to inconsistent tokenizer configurations between training runs. 1 6

More inspectable evaluations—not blanket validation

EvalEval says it published structured records covering 48 benchmarks for the 8B model and 47 for the 70B, including evaluator identities and generation settings. The organization explicitly collaborated with Apertus: its announcement corroborates publication of those records, not independent reproduction of all the scores. 4

The official data-transparency repository provides a versioned catalog of dataset modalities, licenses, versions, storage paths and processing scripts. It also documents internal validation before datasets enter training. That establishes a documented review process, rather than an independent audit of every dataset or proof of legal compliance. 3

Benchmark gains come with trade-offs

The developer’s technical report shows substantial gains over Apertus 1.0 on the following 8B benchmarks, but an uneven comparison with Qwen3.5 9B. These are developer-reported percentages with thinking disabled; IFEval uses prompt-level strict accuracy and HumanEval uses pass@1. 6

Benchmark Apertus 1.0 8B Apertus 1.5 8B Qwen3.5 9B IFEval 68.4 87.1 84.3 Global-MMLU 55.5 59.4 74.6 HumanEval 41.4 68.8 88.3 MATH-500 19.6 51.2 87.2

Source: the Apertus team’s technical report, Table 23. The comparison supports an instruction-following strength, not across-the-board superiority over similarly sized open-weight models. 6

Separate first-hand testing by DS-NLP Lab also found gains and regressions. Using one H100 and a common evaluation setup with a remeasured Apertus 1.0 baseline, the lab reported normal-mode IFEval rising from 68.9 to 85.8 and GSM8K from 60.2 to 79.6. Multilingual HellaSwag fell from 61.9 to 53.6. The lab describes its coverage as non-exhaustive. 7

Thinking is not a universal improvement switch. The developer reports that enabling it raises 8B MATH-500 from 51.2 to 71.4. In DS-NLP Lab’s separate setup, it lowered IFEval from 85.8 to 72.1. These are different tasks and evaluation conditions, but together they support workload-specific testing rather than assuming thinking always helps. 6 7

The developer’s 87.1 IFEval score and the lab’s 85.8 remain unreconciled. The lab discusses sensitivity to metrics, tasks and chat templates, but the available evidence does not establish the exact cause of the difference. The scores should not be averaged or treated as interchangeable replications. 6 7

The release notes characterize Apertus as competitive with leading open-weight models on Swiss legal tasks. The linked SwissLegalEvals results support a more specific conclusion: Apertus 1.5 70B led translation of court-decision summaries with 71.7, while its translation average of 65.2 ranked fifth. Its overall composite was 45.2, compared with 62.3 for the leader, and its legal multiple-choice average was 33.8. 1 8

That is evidence of a legal-translation strength, not broad parity in legal reasoning and summarization. The evaluator also uses LLM judges for generative tasks and reports uncertainty around composite rankings. 8

Deployment limits remain important

The model card says thinking and tool calling cannot be combined, native video input is unsupported, and available memory may require reducing the configured context length. Apache licensing also does not mean ungated downloads: Hugging Face access requests contact-information sharing and acceptance of an acceptable-use policy. 5

The technical report limits its BFCL tool-use assessment to single-turn categories, rather than the full official leaderboard, and says the release was not post-trained for agentic use. It also identifies persistent factual hallucinations, over-refusal, unevaluated long-tail language quality and several multimodal benchmarks on which the 70B trails the 8B. 6

Production evidence is mixed, too. Liip reported a 20-question controlled test in which answers rated good increased from 65% to 75%, while those rated acceptable fell from 100% to 85%. Liip identified retrieval differences and refusal behavior as important factors, called the test non-scientific, and disclosed that Apertus team members reviewed its draft. 9

Serving support requires checking separately. The official ecosystem page lists providers including Swisscom, Infomaniak and PHOENIQS, but dates its information to September and warns that some features require inference-side configuration changes. It does not establish which current endpoints expose every modality, thinking, tool calling or the full context window—or their memory and latency requirements. 10

Documentation is not fully synchronized: the model card still calls the technical report forthcoming despite the preprint’s October 11 submission, and directs users to modified vLLM and Transformers builds. Current upstream support was not independently verified in the underlying research. 5 11

What remains unresolved

Beyond reconciling benchmark scores and checking serving support, external reconstruction of the exact training mixture remains untested in this research. A separate comparative claim in the release notes concerning the Last Translation Benchmark also remains unverified because the linked page exposed methodology but not result rows. 3 1

The practical takeaway is to evaluate Apertus 1.5 against a specific workload: the available tests show meaningful improvements alongside regressions and deployment-dependent outcomes, not a universal upgrade verdict. 7 9

Research scope: This draft uses the supplied documentation and published evaluations through October 16, 2026. It includes no new model execution, independent benchmark reruns, exact training-mixture reconstruction or full training-data and license audit.

Sources

  1. Apertus 1.5 Release Notes huggingface.co
  2. Apertus AI apertus-ai.org
  3. GitHub - swiss-ai/apertus-data-transparency: Versioned archives of key legal documentation from the Apertus releases · GitHub github.com
  4. Apertus 1.5 sets the bar for open, reproducible evaluation reporting | EvalEval Coalition evalevalai.com
  5. swiss-ai/Apertus-v1.5-8B · Hugging Face huggingface.co
  6. From Open Text Models to Open Multimodal Reasoners Apertus v1.5 Technical Report arxiv.org
  7. LLM Benchmark Evaluation - Apertus 1.5-8B | DS-NLP Lab blog.nlp-lab.ai
  8. SwissLegalEvals/blog/swiss-legal-evals-2026.md at main · JoelNiklaus/SwissLegalEvals · GitHub github.com
  9. Apertus 1.5 - first impressions from using Switzerland’s updated AI model · Blog · Liip liip.ch
  10. Apertus AI apertus-ai.org
  11. [2610.15542] Apertus 1.5: From Open Text Models to Open Multimodal Reasoners arxiv.org

Numbered citations refer to the sources listed for this article.

More in Models