Version 2.20.5

2026-07-21

LLM judge evaluation, invitation codes, credits, benchmarks from Excel, and a broad security update.

New

LLM judge (LLM-as-Judge) evaluation: in multiple-choice and question-answer benchmarks the model's answers can also be scored by a judge model.
Invitation codes: administrators create and manage them, users can request one.
Credits: every submission uses credit, and users can request more.
Create multiple-choice and question-answer benchmarks from an Excel file, with a preview.
Image benchmark type.
Login with SSYZ / e-Devlet.
Submissions run one after the other; waiting ones show as "Queued".
Leaderboard sorting, filtering and download as CSV or JSON.

Improvements

Large model files are uploaded in parts, more reliably.
The whole interface is translated to Turkish; error pages are in Turkish.
Login, session and file upload security strengthened; an account is locked for a while after many failed logins.

Fixes

A stuck submission no longer blocks the queue; it is recovered automatically.
Files with Turkish characters in their names download correctly.

Edit Created By: Key: Is Public: Yes No Files Solutions Search by name... Show Public Tasks Upload Task Create Task Delete Selected Tasks Name Description Creator In Use Public Actions Select Users Select top 5 biggest users Download as CSV Last snapshot date: No snapshot has been taken yet User Joined at Datasets Submissions Total usage Benchmarks usage Users usage Administration usage Orphaned files usage Select top 5 biggest benchmarks Benchmark Organizer Creation date Submission Management Search Users... Search Benchmarks... Re-run Selected Stop Selected Delete Selected File Name File Size Submitted at Processing Start Processing End Compute Worker Queue Docker Image User Management Portal Search Users... Name Username Email User Type Delete User Delete User ? Cancel Change User User ID Switch User Search... Delete Selected File Name Benchmark Size Uploaded Delete? No Datasets Yet! Details Key Created By Created Description: Used by: Download File Close Filter Leaderboard by Columns Start typing to filter columns under 'Meta-data' or Tasks. Task: # Participant Date ID This CSV This JSON You are not a participant of this benchmark. Please register in My Submissions tab to view the leaderboard. Your request to participate in this benchmark is waiting for an approval from the benchmark organizer. Your request to participate in this benchmark is denied. Please contact the benchmark organizer for more details. No submissions have been added to this leaderboard yet! Preparing submission... this may take a few moments... Scoring Ingestion ---- Approved Pending Denied Unknown Show deleted accounts Email all participants Download all participants Username Is Bot? account deleted Revoke Approve Send Message Send Email Subject A message from the admins of {competition_title} Content Number of submissions used for the day The submission limit resets at midnight server time. Number of total submissions used All ID # File name Owner Phase Date Status Score Detailed Results Actions Run Submission Rerun Submission Delete Submission Cancel Submission Add to Leaderboard On the Leaderboard Make Public Make Private Task Admin ERROR: Submission is a parent, but has no children. There was an error during creation.