We will be further increasing matches played for a few hours today, but I believe we are still on track for the 5pm cutoff. Our main concern was that algos uploaded the night before the deadline would be unfairly unable to reach the top 10 in the time provided, even with very high winrates. With top players ‘last minute’ algos reaching fairly close to their pre-existing algos, and with last minute algos like some by @zigzagninja and @RuberCuber reaching the top 10.
Ultimately, from my perspective, our system seems to be working pretty well given the constraints of the Elo system, and the top 20 look fairly close to what we expect them too. In my view there seem to be two problems causing the issues addressed here:
We will be announcing it officially during the stream, but we will be switching to the glicko2 system next season. You can read about it more online, but it should fix nearly all of the issues Elo causes and increase the overall quality of the rating system significantly. This will prevent large jumps in score for single game losses, ‘tightening in’ algos with consistent performance, and dramatically reducing the time it takes to reach a score close to what your algo should have.
A more fundamental problem is that we have a dozen extremely talented engineers competing for the ‘cutoff’ spots who have invested a significant amount of time into terminal and have created some great algos, and only a small handful of them will be in the competition. I think hard cutoffs like this where #10 is ‘in’ and #11 is ‘out’ are fundamentally problematic, but smoothening the curve between ‘ins’ and ‘outs’ is a very difficult problem. Many of you know we have slowly but surely been building out our platform to support players connecting with our partners online, similarly to how they might at a live competition, so this might help at some point.
In any case, I consider our teams willingness to experiment and learn from problems that come up one of our strengths, so we will at least have some discussions about what we can do to improve our large-scale competitions.
That was a bit longer than expected and still doesn’t actually cover everything i’m thinking about, but feel free to ask questions or continue the discussion below. Just to clarify, we are currently expecting to go forward with the 4pm cutoff time.