Overview / Usage

MADRaS is a multi-agent extension of Gym-TORCS and is open source, lightweight, easy to install, and has the OpenAI Gym API, which makes it ideal for beginners in autonomous driving research. It enables independent control of tens of agents within the same environment, opening up a prolific direction of research in multi-agent reinforcement learning and imitation learning research aimed at acquiring human-like negotiation skills in complicated traffic situations—a major challenge in autonomous driving that all major players are racing to solve.

Most open-source autonomous driving simulators (like CARLA, DeepDrive, AirSim) innately support only egocentric control; that is, single agent behavior, and have preprogrammed behaviors for the other agents. The difficulty in introducing agents with custom behaviors in these simulators restricts the diversity of real-world scenarios that can be simulated. To address this issue, we developed MADRaS, wherein each car on the racing track can be independently controlled, enabling the creation of rich, custom-made traffic scenarios, and learning the policy of control of multiple agents simultaneously.

Methodology / Approach

The project extends the popular open-source simulator TORCS, which is essentially a car-racing game wherein a player controls a car to finish the required number of laps around pre-designed racing circuits first, amongst tight competition with the other racing cars, which are computer-controlled and heuristic-driven.

MADRaS extends a wrapper around this game simulator which enables multiple cars to be controlled via coded scripts as opposed to keyboard inputs. What is novel about this is that one could use learning algorithms in these scripts to control the multitude of cars, which, as mentioned previously, is a critical avenue of research for autonomous driving researchers. Different agents connect to the server via different UDP ports, and independently communicate actions and receive new inputs from the server. In order to prevent lags, the server uses a time-out interval which re-uses the last action sent by an agent if it fails to send a new one within a pre-specified time interval. This encourages the agents (which could be using neural networks) to compute every next action quickly, which reflects the fast and reactive driving requirements in the real world.

Moreover, since MADRaS is built on the TORCS graphics engine, one can hand-design a large range of custom scenarios by creating new tracks quickly and easily using the available 'trackeditor' and 'trackgen'. Additionally, the driving behavior of each of the other 'traffic' cars on the road can the customized by a few edits in driving logic of individual cars, and one could also use the aforementioned external scripts to define hard-coded behaviors of the cars connected via the UDP ports. Both of the above features enable a vast variety of scenarios to be created and tested for autonomous driving in the multi-agent setting!

Technologies Used

Running MADRaS has minimal hardware requirements, with no need for a GPU. It works even on 5th generation Core i3 laptops. With that being said, heavier simulations and experiments involving five or more agents will run faster and smoother on powerful machines having faster Intel processors with a large number of threads. For designing custom tracks using heavy-weight softwares like Blender, one might use powerful machines with a GPU.

In terms of development, the project was developed and optimized entirely on the Intel® Core™ i7 processor and Intel® Xeon® processors, while running smoothly on other x86 platforms as well. Currently, integrating MADRaS with the Intel® Nervana™ platform Reinforcement Learning Coach is work-in-progress, and we invite the community to participate in its development.

Repository

https://github.com/abhisheknaik96/MultiAgentTORCS

Other links

• Official Blog
• Media coverage