We will be further increasing matches played for a few hours today, but I believe we are still on track for the 5pm cutoff. Our main concern was that algos uploaded the night before the deadline would be unfairly unable to reach the top 10 in the time provided, even with very high winrates. With top players ‘last minute’ algos reaching fairly close to their pre-existing algos, and with last minute algos like some by @zigzagninja and @RuberCuber reaching the top 10.
Ultimately, from my perspective, our system seems to be working pretty well given the constraints of the Elo system, and the top 20 look fairly close to what we expect them too. In my view there seem to be two problems causing the issues addressed here:
Elo System
We will be announcing it officially during the stream, but we will be switching to the glicko2 system next season. You can read about it more online, but it should fix nearly all of the issues Elo causes and increase the overall quality of the rating system significantly. This will prevent large jumps in score for single game losses, ‘tightening in’ algos with consistent performance, and dramatically reducing the time it takes to reach a score close to what your algo should have.
A more fundamental problem is that we have a dozen extremely talented engineers competing for the ‘cutoff’ spots who have invested a significant amount of time into terminal and have created some great algos, and only a small handful of them will be in the competition. I think hard cutoffs like this where #10 is ‘in’ and #11 is ‘out’ are fundamentally problematic, but smoothening the curve between ‘ins’ and ‘outs’ is a very difficult problem. Many of you know we have slowly but surely been building out our platform to support players connecting with our partners online, similarly to how they might at a live competition, so this might help at some point.
In any case, I consider our teams willingness to experiment and learn from problems that come up one of our strengths, so we will at least have some discussions about what we can do to improve our large-scale competitions.
That was a bit longer than expected and still doesn’t actually cover everything i’m thinking about, but feel free to ask questions or continue the discussion below. Just to clarify, we are currently expecting to go forward with the 4pm cutoff time.
Ok, interesting, thanks for noticing these problems and trying to fix them at such short notice. Im a little confused now though is the cutoff at 5pm or 4pm?
If that’s the elo system I’m thinking of, that Halite uses, I’m a huge fan. They announced some details about their rating system for Halite III that I’ve always felt it was a fantastic system for this type of competition. I’m not sure how 1 upload vs 6 uploads changes things, but I’m eager to see this implemented for Terminal.
Goodness. I’m looking at my algos right now and it seems whoever’s on top has just had the most recent win streak vs 1700 elo algos lol. Two peaked at 2110 doing that it seems. You can see a very visible jump in elo across the board from the time more matches/bigger pool of elos got turned on. I’m curious if I briefly held the top spot in the last 24 hours; I’ve been checking sporadically and have been floating around 2nd-4th but it’s always a different algo when I check.
Big shout out to winners and most importantly those who did not make it even though did their best!
I had fun to be part of the top 10 for some of the time but ended up as 14th.
As @RegularRyan mentioned ‘cut off’ is rather harsh but looking at other examples, e.g., sports there are only 3 sets of medals. This is much more democratic.
Greetings. I’m representing 16daystocode (more on that below).
Congrats to all of the other 10 ten finishers as well as everyone else! Will we be getting any feedback on which bots will be in the finals before the matches occur? One of our algos was tied for 9th and 10th place at the end but shortly after (roughly 8 minutes later) another one of our algos overtook it. Our highest algo at the time of the 4pm ‘snapshot’ is the algo that we would like to use. We just want to make sure that is correct.
As a side note, we cant seem to activate a forum account for 16daystocode. The link provided to verify our email address doesn’t work, even when we copy and paste it.
Will we be getting any feedback on which bots will be in the finals before the matches occur?
Dues to timezones, I was asleep when the cutoff happened, since my algo is near the bottom of the top 10, I have no idea whether it made the cutoff or not. A list of the top 10 at the cutoff point would be nice.
When we stopped the servers, it took a few minutes for the servers to actually stop running matches. We are using the leaderboard as you see it in its current state for the competitions. Fortunately, the same 10 players are in your screenshot and the current leaderboard, so hopefully nobody is to upset about the re-ordering.
If multiple players get the same number of wins, the winner will be the player with the most wins amongst that subset of players. Further ties follow that step recursively until no players are removed from the subset. In such a case, the highest Elo player will move up.
Sorry if I did not communicate the above, I thought it was in the details above, but it is described incorrectly as we did not account for 3 way ties in the round robin group. Ill just remove it for now.
Sorry about reduced communication on an important day for terminal, i’m on vacation rn.
