It’s game on for
OpenAI, the research initiative co-founded by
Elon Musk that has been setting its
AI bots against amateur
Dota 2 players and winning.
Next up, OpenAI will be challenging the top players of the arena battle game at the Dota 2 tournament The International, described by The Verge as the biggest annual event on the e-sports calendar. Indeed, professional and highly motivated gamers train year-round to get a shot at Dota’s annual $40 million prize pool.
What’s going on here with OpenAI: the lab has already successfully trained its bots to beat top Dota 2 players in head-to-head games. What the lab is trying to do now is set them up to take on human teams in competitions of 5-against-5.
It is, it should go without saying, a massively complicated undertaking. OpenAI teaches the AI bots through reinforcement learning, putting them into a virtual world and using trial and error to help them figure out for themselves how to reach a goal. Reward functions are identified, where the bots score points when they accomplish a certain task, but other than that they’re basically set loose to play themselves.
For this latest round of training, OpenAI’s bots played the equivalent of 180 years’ worth of games against themselves every day. OpenAI co-founder and CTO Greg Brockman told The Verge it starts out random, such as by doing something like wandering around the map. Basic skills are picked up in a matter of hours. What it all amounts to — if you assume that it takes a human at least 12,000 hours to become a professional at the game, OpenAI’s systems “play 100 human lifetimes of experience every single day.”
The OpenAI bots have plenty of advantages that you can guess. Faster reaction times, for example, plus instant access to data like inventories and heroes’ health that human players would have to check manually. To be sure, they’re also not getting a taste of the fullness of Dota 2, with limitations set that include only making use of five of 115 heroes — that all have different playing styles – that are available. Some aspects of the game like invisibility have been disabled completely.
OpenAI Five — a team of five neural networks — “averages around 150-170 actions per minute (and has a theoretical maximum of 450 due to observing every fourth frame),” reads an OpenAI blog post published today about the news. “Frame-perfect timing, while possible for skilled players, is trivial for OpenAI Five. OpenAI Five has an average reaction time of 80ms, which is faster than humans.”
All of this is not just for fun or bragging rights. There is certainly an application here that can prove useful in the future. An AI system that can learn how to master a complicated video game could be put to use in the real world, for example, such as by optimizing something like a municipal utility grid.
“Our underlying motivation reaches beyond Dota,” continues the OpenAI blog post. “Real-world AI deployments will need to deal with the challenges raised by Dota which are not reflected in Chess, Go, Atari games, or Mujoco benchmark tasks. Ultimately, we will measure the success of our Dota system in its application to real-world tasks.”
Leading up to the August competition, OpenAI is planning to host a match with top players on July 28. You can follow along on Twitch to watch live or request an invite to attend in person.