Version 2.20.8

Turkish answers are scored correctly; the Twenty Questions benchmark gets a fixed answerer, a clear score formula and time limits.

New

• In the Twenty Questions benchmark every player plays against the same fixed answerer. Your model sees only the instructions and the dialogue; the secret word never reaches it.
• Twenty Questions has time limits per turn, for the answerer and per game. The rules page says what happens when time runs out; results show the time of every turn and game separately.

Improvements

• The Twenty Questions score is clearly defined: a lost game scores 0, a won game 60 + 40 × efficiency; the overall score is the average over all games. The formula and a hand-checked example are on the scoring page.
• The Twenty Questions answerer answers basic questions such as "Bir meslek mi?" correctly.
• When creating a benchmark from Excel, Turkish column headers in capitals ("KATEGORİ", "DOĞRUCEVAP") are recognised.

Fixes

• Turkish answers are scored correctly: capital and small letters (İ/i, I/ı), Turkish quotation marks and apostrophes no longer make a right answer wrong. "İSTANBUL" and "İstanbul" are the same answer; "kır" and "kir", or "şehir" and "sehir", are not.
• In multiple-choice questions, replies such as "Doğru cevabı B şıkkıdır.", "CEVAP: D" or the choice itself ("İZMİR") are read correctly. In the SSB benchmark, "CEVAP: D" was counted as A.
• Category names are capitalised with Turkish rules ("İletişim").
• Benchmarks created from Excel earlier get these fixes when they are created again from their Excel file.