My simulator should hopefully be complete soon, but currently, it simulates everything as long as my units can score. If there is no way to score, my algo crashes (times out) because of my current method for pathing. The fix should be fairly easy, but I donāt have any time because of how many assignments I have for school right now 
.
As for polling for optimization, firstly, assuming that everybody is using python (which they arenāt, but I think that most people are), I would recommend using timeit for measurement. It is extremely easy to use. However, this misses some aspects of optimization. My simulator, for example, takes roughly twice as long for the first simulation. The way that I do pathfinding means that initializing it takes some time, but it can be reused for many simulations, assuming that nothing major happens to the map between simulations (If I were to test what would happen if I made a maze in one turn, for example).
Because of this, I would say that a couple of options are:
- Time that it takes for one simulation
- Time that it takes for 100 simulations (optimizations, such as memoization, allowed)
- Number of simulations that can be completed in 5 seconds (optimizations allowed)
If the time measured is the time for one simulation, one possibility is to check the time for 100 simulations anyway, but from the beginning (no memoization, saving info, etc.). I have found that if I am measureing an extremely short amount of time with timeit, it sometimes takes non-negligible time itself. To be fair, this is usually when I place it in get_target, because it is the limiting factor. In any case, to ensure that this isnāt an issue, something like this could be done:
start_time = time.clock()
for _ in range(0, 100):
<indent> # Simulate, insure no information is reused
end_time = time.clock()
Another consideration is the match played. Depending on how long units survive and the density of firewalls (for pathfinding), it would be another factor to consider. I believe that the easiest way to do this would be to use the starter algo for the strategy, and play it against the starter algo as well. A similar issue is which turn the simulation is completed on. A starter algo vs. starter algo match takes 17 turns, so the time for the first 16 (in case the end is strange) turns, or to take the cumulative time for these rounds. However, in 17 rounds, I donāt think that the firewalls become very dense, so maybe another method is better? Maybe playing the starter algo against a specific boss, or something like that?
Something else to consider is the placement of units for the simulation (my simulator is very fast if it assumes no information units are placed 
). A suggestion, for the sake of simplicity, is to assume the placement that the algo will do next turn, which can be known for certain due to the use of the starter algo on both sides (hopefully).
Another consideration is where the game is played from. I think that the best way would be to upload the algo and play it on the playground, assuming the playground uses the server, and not the local computer. I donāt actually know whether it does. Does anybody else? It might be necessary to check time in ranked matches, which is another issue, as the opponent could not be controlled.
Did I miss any other factors to consider?
as a rule of thumb I assume anything I hand write in Python is 10x slower than a C++ equivalent