Version 2.20.7

2026-10-06

Clearer error messages, image benchmarks with categories, and pages that fit on phones.

New

The reason a submission failed is now shown: an error type and a short explanation. Click the ? next to the submission for the details.
A model that cannot run in this environment is reported, with the reasons, before the submission is queued.
Questions of an image benchmark can be grouped into categories. The organizer can set each category's weight in the overall score; the leaderboard has a column per category.
Multiple-choice benchmarks can have category weights too, and the organizer fixes the benchmark's answer mode (with reasoning or answer only).
In the model template, a separate vision model can have its own address and key.
This page: see what changed in every update.

Improvements

Pages fit phones and tablets without scrolling sideways.
Empty screens (no submissions or results yet) show the next step.
English text left in the Turkish interface is translated; terms are used consistently.
The model upload form explains how to prepare the archive; an archive with the wrong structure is rejected with the reason.

Fixes

A model's reply was counted as a choice letter far too easily ("I don't know" was read as B). Now only a reply that clearly names a choice counts.
The benchmark links in participation emails and the links in forum emails work.
"Invalid DateTime" in some dialogs and half-translated text are fixed.

Edit Created By: Key: Is Public: Yes No Files Solutions Search by name... Show Public Tasks Upload Task Create Task Delete Selected Tasks Name Description Creator In Use Public Actions Select Users Select top 5 biggest users Download as CSV Last snapshot date: No snapshot has been taken yet User Joined at Datasets Submissions Total usage Benchmarks usage Users usage Administration usage Orphaned files usage Select top 5 biggest benchmarks Benchmark Organizer Creation date Submission Management Search Users... Search Benchmarks... Re-run Selected Stop Selected Delete Selected File Name File Size Submitted at Processing Start Processing End Compute Worker Queue Docker Image User Management Portal Search Users... Name Username Email User Type Delete User Delete User ? Cancel Change User User ID Switch User Search... Delete Selected File Name Benchmark Size Uploaded Delete? No Datasets Yet! Details Key Created By Created Description: Used by: Download File Close Filter Leaderboard by Columns Start typing to filter columns under 'Meta-data' or Tasks. Task: # Participant Date ID This CSV This JSON You are not a participant of this benchmark. Please register in My Submissions tab to view the leaderboard. Your request to participate in this benchmark is waiting for an approval from the benchmark organizer. Your request to participate in this benchmark is denied. Please contact the benchmark organizer for more details. No submissions have been added to this leaderboard yet! Preparing submission... this may take a few moments... Scoring Ingestion ---- Approved Pending Denied Unknown Show deleted accounts Email all participants Download all participants Username Is Bot? account deleted Revoke Approve Send Message Send Email Subject A message from the admins of {competition_title} Content Number of submissions used for the day The submission limit resets at midnight server time. Number of total submissions used All ID # File name Owner Phase Date Status Score Detailed Results Actions Run Submission Rerun Submission Delete Submission Cancel Submission Add to Leaderboard On the Leaderboard Make Public Make Private Task Admin ERROR: Submission is a parent, but has no children. There was an error during creation.