The CodeBullet challenge stream is being delayed until 7pm tomorrow, and competition rounds are going to start going out tonight or tomorrow morning. All rounds other than the final 3 will be released before 7pm tomorrow. The final 3 rounds will be part of the stream tomorrow.
This is our first time running a competition on this scale - we did a few dry runs with 700 very simple Algos in our development environments, and ranked matches have been going smoothly, so we were confident that we would be able to run this contest without difficulty. Unfortunately, it looks like the higher complexity and diversity of our user submitted algos and the nature of the āliveā environment had some unexpected impact on our tournament system.
A number of matches have been getting messed up, and we can reset them by hand, but it is a slow process. Ultimately, we want to ensure the quality and integrity of all the matches that take place, and will be spending an extra day to ensure that tomorrows stream runs as smoothly as possible.
Sorry about the delay, and for failing to notice the extent of these issues early enough to provide better notice about the reschedule.
Elo doesnāt matter in the competition.
They play single elimination brackets and most algos go straight into round 2 because the number of players is not a power of 2.
What algos play against each other is completely random and which algos skip round 1 is also random.
Then they play until 8 algos are remaining and the matches are released in the āmatchesā tab in the competition.
The final matches of the top 8 algos will not be released yet, they will be shown at the stream and get released later.
You can be lucky with an algo that counters a few strategies but it is more likely to win with an algo that counters most strategies at the same time so I think the best algo also has the highest chance to win.
You are right that a simple algo can be really lucky and win by chance (i.e. if top algos encounter algos that are made to counter that specific algo)
But they know this and they already confirmed they are working on other systems than single elimination brackets for the future.
Given the number of entrants, I believe there will be 11 rounds:
630 -> 512 -> 256 -> 128 -> 64 -> 32 -> 16 -> 8 -> 4 -> 2 -> winner
Even with the first round bye, thereās no way a narrow strat is going to run the table. Unless that strategy is just straight up the best, then I wouldnāt call that luck and congratulate the person who refined it.
Given the number of rounds, I would say that āwinning by luckā is pretty unlikely. The odds of getting matched against algos that you counter, if you only counter a few, gets exponentially unlikely.
However, I would say that there may be such a concern as ālosing by luckā, if you have an algo that beats most opponents, but gets countered by one specific strategy, you only need to run into that strategy once in any round and youāre out of the running, no matter how many other strategies your counter loses to.
We use the single round elimination bracket because we feel it creates an exciting, easily understandable, and immediately familiar tournament structure. Other structures trade off these features for improved āaccuracyā, but ultimately still have some degree of luck involved. While we do plan to experiment with different structures in the future, we have historically seen a strong correlation between competition standings and Elo.
The āluckā factor is naturally increased in a competition where only 1/600 people will take home the first place prize, but we are still confidant that we will be seeing our top players make it much farther more consistently than other players.
