On Wednesday, a team of researchers from the University of Zürich and Intel announced that they have developed an autonomous drone system named Swift that can beat human champions in first-person view (FPV) drone racing. While AI has previously bested humans in games like chess, Go, and even StarCraft, this may be the first time an AI system has outperformed human pilots in a physical sport.
FPV drone racing is a sport where competitors attempt to pilot high-speed drones through an obstacle course as fast as possible. Pilots control the drones remotely while wearing a headset that provides a video feed from an onboard camera, giving them a first-person view from the drone’s perspective.
The researchers at the University of Zürich (UZH) have been trying to craft an ideal AI-powered drone pilot for years, but they previously needed help from a special motion-capture system to take the win. Recently, they came up with an autonomous breakthrough based largely on machine vision, putting the AI system on a more even footing with a human pilot.
Official video for “Champion-level Drone Racing using Deep Reinforcement Learning” (Nature, 2023).
Their solution, Swift, uses real-time data collected by an onboard camera similar to those used by human racers. It also incorporates an integrated inertial measurement unit that measures the drone’s acceleration and speed. An artificial neural network processes this data to localize the drone in space and identify race gates on the course. This information is then used by a control unit, which is also based on a deep neural network, to choose the best course of action to complete the race circuit as quickly as possible. The team trained the AI model using reinforcement learning in a simulated environment, where the system taught itself by trial and error.
Advertisement
Swift competed against three world-class human pilots: the 2019 Drone Racing League champion Alex Vanover, the 2019 MultiGP Drone Racing champion Thomas Bitmatta, and three-time Swiss champion Marvin Schaepper.
Regina Sablotny / UZH
The races occurred between June 5 and June 13, 2022, on a specially designed track that covered a 25-by-25-meter area and featured seven square gates that the drones had to pass through in a specific order to complete a lap. The circuit also included challenging maneuvers such as a “Split-S,” which is “an acrobatic feature that involves half-rolling the drone and executing a descending half-loop at full speed,” according to UZH.
Swift secured multiple wins against the human champions and even recorded the fastest lap, finishing with a half-second lead over the best human pilot. However, it was noted that the AI system showed limitations in generalization and adaptability, struggling when the conditions differed from those it was trained for, such as changes in lighting. Overall, the contest demonstrated that while AI has made significant strides in navigating physical environments, human pilots still maintain an edge in adaptability and responding to variable conditions.
The research team behind the new drone AI system includes Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. They published their results on Wednesday in a Nature article, titled “Champion-level drone racing using deep reinforcement learning.”
According to the team, this new technology has several potential real-world applications. Scaramuzza, the head of the Robotics and Perception Group at UZH, says that flying drones faster increases their utility due to their limited battery capacity. Swift’s capabilities could find applications in forest monitoring, space exploration, and the film industry, where fast-moving drones could shoot action scenes. Additionally, the technology could be helpful in search-and-rescue missions where drones need to cover large areas quickly.
While surveying the social landscape online, it’s clear that UZH’s work with drones has been impressing others. On Hackernews, a user who identified themselves as a consultant who has developed computer software for drone racing in the past said, “So glad to see this team from UZH continued pushing the envelope and are now beating human champions. If you saw the team and what they managed in under a year, it was clear they were highly talented and human racers had their work cut out for them to stay ahead.”
