Unlocking the potential of deep reinforcement learning
As Voyage’s CTO, I spend a lot of time thinking about how we can achieve the next breakthrough in our self-driving technology. As the technical progress in autonomous vehicles drives forward, there is an increasing need to explore and build data-driven algorithms that can handle the complexity of driving, and where heuristic rule-based algorithms fall short. At Voyage, we believe there are countless engineers who could be positioned to help tackle these challenges. Unfortunately, to be able to work effectively on these problems, it requires having the right tools or access to a vehicle which is mostly limited to the few engineers that are working at self-driving car companies.
Our team has been quietly working on a project and now we are thrilled to unveil it. Today we are announcing the release of Voyage Deepdrive — our free and open-source self-driving car simulator. To lead this effort, we have brought Craig Quiter on board. Craig is a long-time open source developer in the industry and he’s also the creator of Deepdrive.io upon which Voyage’s platform is based.
The story of Deepdrive goes back quite a few years when Craig — inspired by the progress made in AI and the positive impact a self-driving car could have on society — started the project as an open-source platform to enable anyone in the world to work on self-driving technology.
Deep Reinforcement Learning
I had the pleasure of working with Craig in previous self-driving vehicle companies and got to see his passion and commitment first hand. We often discussed what it would take to truly unlock the potential of deep reinforcement learning, or other end-to-end methods, in the field of autonomous driving. We both agreed that what his simulator lacked (and other similar ones) was a direct connection back to a self-driving car company that intimately knows the hardest problems to solve. We also felt that it was important that these types of simulators achieve much higher levels of adoption to tap into the expertise out in the community. So we decided Craig should join Voyage full-time to leverage his existing work to build out what we are launching today as Voyage Deepdrive.
At the heart of Voyage Deepdrive is a focus on end-to-end learning and deep reinforcement learning. At Voyage, we use simulation based development working with Applied Intuition, and that has driven our core production software forward. While this simulation use will continue to be our main development workflow, Deepdrive provides us a sandbox for research and exploring academic approaches. The reinforcement learning technique — which involves learning by trial and error — has shown tremendous promise in other complex tasks. Deepdrive’s first class support for reinforcement learning means that researchers can focus solely on the problem of driving rather than worrying about the stack required to run their agents on a physical car. We hope this will result in significant productivity gains for researchers wishing to iterate quickly on their ideas in this area.
By launching Voyage Deepdrive, we want to democratize self-driving car research by decoupling the algorithm development from the hardware. Our vision is to provide these tools to anyone who could benefit from them, while ensuring that their work can really benefit and progress the field.
We do this by making sure that the simulator is guided by requirements from a real self-driving car company that faces these challenges everyday. We want to build worlds, scenarios, and events that have been historically hard to solve. We want to build an experimental sandbox for exploring deep reinforcement learning and other machine learning approaches that we hope can push the state-of-the-art in self-driving cars forward. And we want to reduce the barriers to entry so we can attract many more engineers to the field.
As part of the release of Voyage Deepdrive, we are introducing some new and notable features. We are particularly excited about an ongoing leaderboard where teams and individuals can see how their agent stacks up against the rest. The simulator features a beautiful and accurate 3D world built by Parallel Domain, where a rich environment allows the testing of code to be more realistic. We have also introduced a feature where users can not only generate their own agents, but also generate new driving scenarios for agents to be tested against, which will help lead to more sophisticated algorithms.
https://medium.com/media/2ebbd3041135dfb460879d3e51b2a425/href
Get Started!
In the coming months, Voyage will be hosting a series of competitions to encourage independent engineers and researchers to identify AI solutions to specific scenarios and challenges that actual self-driving cars face on the roads. We hope you’ll follow along as we seek to accelerate the development of the AV industry and to push the boundaries of the performance, capabilities, and safety of self-driving cars.
Head over to Voyage Deepdrive to get started.
Introducing Voyage Deepdrive was originally published in Voyage on Medium, where people are continuing the conversation by highlighting and responding to this story.