Computation time reduction?

[arnby]

Hi everyone!

Yesterday during the final I said few things about this, this post is just to get into the details.

Aside from messing around with ML algos, I was pretty much away for season 4. About a week before the deadline, I decide to put back my Genoracle algo online. I was a bit surprised because the decisions it made had an unfinished color, some moves were clearly sub optimal (like sending lone pings, which has also been seen during the final). So I dig a little to inspect this.

I remembered that C1 said that for season 4 computation time will be better regulated. As my strategy is very hungry in computation power, I assumed it would affect me (it was very likely that my algos were using the opponent’s time as they said). At the end of season 3, my algo could do between 40k and 120k simulations per turn and was expecting for it be divided by at most 2 with the season 4 system. So I was quite surprised when I noticed that it was now between 10k and 30k.

So I looked back at my old posts and simulators to find some numbers.

On January 2019, my algo was computing 9k simulations per turn on my computer and 40k on the server.
Today (January 2020), my algo can compute 80k simultations per turn on my computer. But it can only achieve 20k on the server.

This confirm what I remembered: The server, in past seasons, had always outperformed my computer in computation powers. So I was a bit surprised that it reversed completely.

Has anyone also noticed this? Or I am just finding excuses for the losses of my creations ?

On a side note, as it is now, the strategy behind Genoracle is unsustainable. I supposed it is my fault for making such a computational heavy strategy.
For the final I named it “Genoracle_is_not_dead” as a wish for it to have some kind of last show and I’m glad it made it in!

Anyways, it was so interesting to see the beautiful algos you all made this season! Thanks everyone for the effort and talent you put in this . It is always a learning experience to see your creations playing!

[acshikh]

I have an i7-6820HQ, and I consistently see that the server does only about half the number of simulations as my laptop does, which is fairly inline with what you seem to be getting now after the rebalancing.

[C1Junaid]

So the promise of better computation time consistency for Season 4 kind of failed because the approach we used with Docker caused all sorts of problems so we had to revert it.

So computation should be the same throughout the seasons on paper.

What I suspect could be happening is that the algos you are versing hog more/less CPU. If this is the case we could tell by seeing how many simulations happen between a ranked game with a “simple” algo, and a ranked game with a “high performance” algo. If the performance even against a “simple” dumb algo is bad then maybe there really is something wrong and we can spend some time looking at the settings. If you already have this data thats great too!

How do you know how many simulations your algo is doing in ranked matches though since you can’t read error logs unless it crashes? If these are the numbers from Playground matches, then that doesn’t really matter unlike our ranked matches Playground servers are essentially big servers that handle dozens of matches each so is super variable the compute you get as its based on the traffic and how busy playground is.

[max1e6]

Would it be possible to add some sort of way for algos to log some statistics like the number of simulations run? Logging statistics could be useful not just for debugging issues like this one, but also for improving our algos

[C1Junaid]

It would be a fairly complicated feature, but something I will think about, maybe there is an easier way.

For now, you could always just have your algo crash if it wants to report something, and then you can read your error log that way Just be careful since algos stop playing matches after a certain number of crashes.

[max1e6]

That’s a fun idea, I can definitely find a use for that for at least some parameter tuning. If I only crash with like 10% probability then hopefully my algo could reach high-enough elo anyways that it plays some matches against good opponents.