The Car That Learned to Drive Itself: AI, Neuroevolution, and Autonomous Driving Simulation in the Browser
Is it possible for a car to learn how to drive on its own, without it explicitly being taught how to drive? This demonstration began as a thought experiment: could a very basic smart car with only a few sensors learn to drive on its own in a simplified sandbox, purely by machine learning techniques and a reward and punishment system?
In order to find out, a neural network class was written in JavaScript from scratch, as well as a sandbox world with an infinite straight road. Basic rules of the road and a collision detection system were implemented, and the software was given a graphical user interface for better visualization. The player car was assigned a compact, fully connected artificial neural network with 10 input neurons, two hidden layers containing 14 and 10 neurons, and four output neurons.
Instead of a conventional back-propagation, a neuroevolution system was implemented, where successful behavior survived and evolved into the next generation, while costly behavior died out. The most interesting results, however, were not the successful ones from a driver's point of view. They were rather the ones in which the car tried to find unconventional solutions, which were mathematically effective, but not expected road behavior.
These false solutions forced constant adjustment of simulation rules, as well as rewards and punishments. In the end, it was demonstrated that a smart car can in fact learn to drive using a neuroevolution system, but the most important demonstration in this simulation was that the machine learning system did not necessarily learn the path the designer had intended. It learned what its objective function rewarded. The model, even though successful in the initial simulation, was extremely overfit and performed poorly in new simulation environments.
1. Introduction
Self-driving cars will be an inseparable part of our daily lives, either as a driver or passenger of one, or as a driver of a traditional car that encounters one, or as a pedestrian. Self-driving cars are here and they are not going anywhere. Therefore it is very important to at least have a basic understanding of how they work and the safety protocols concerning the so-called fully Autonomous Driving Systems.
Around 94% of accidents are attributed to human drivers, 2% are related to vehicles, 2% to the environment and the cause of the rest is unknown.[1] It is not only important to understand the nature of human errors that lead to accidents, but also the unknown causes. For self-driving cars it is absolutely vital to understand the accidents caused by the environment, the vehicle and other unknowns. Scenario-based simulations and synthetic data can complement the real-world data, because they enable creation of challenging situations and their evaluations without risk.[2]
Since we have established the importance of simulation, the next step would be to set simulation goals. We need to make clear what we want to simulate, why we want to do it, and how to get there. The point of this project was to see whether in a limited sandbox world with basic rules, can a car learn to drive without external help. The reason for it, beside discussion on the feasibility of this project, is to learn how machine learning algorithms come up with unforeseeable solutions, and how they need to be addressed. The way to achieve this goal was decided to be a fully connected neural network.
2. Realization of autonomous driving
Machine learning aspect of self-driving cars is not a new topic and has already been studied extensively. In their study, Bojarski et al. trained a convolutional neural network (CNN) to map inputs from a single front camera pixels to steering commands with minimal human input. The model learned to drive on its own without the team having to explicitly teach the model the outlines of the roads.[3]
Real-world deployment of models without proper testing and simulation can be costly. In addition, in terms of safety, the risks are far too high. Aside from safety, simulation opens the door to rapid development of the model.[4]
2.1. Artificial intelligence and simulation
Once the study goals are set, the training data for the system need to be prepared. If the goal is to step beyond just simulation, real-world data is necessary. The training data can consist of hours of driving footage[3] or even images from a driving simulation.[5] Another path towards training a model would be reinforced learning, where based on certain actions of the vehicle, rewards and penalties are defined.[6]
Based on the project requirements, a suitable simulation environment should also be considered. Famously the simulator provided by Udacity can be used as an example in certain cases.[7] For this demonstration, it was decided that a web-based simulation was the best course, as it provided a simple platform to build upon for this limited project. It has already been demonstrated that implementation of artificial neural networks for two-dimensional self-driving with moving and still objects is feasible.[8]
3. Implementation
This project was implemented using JavaScript in the browser to make it accessible across all modern platforms. That also meant sacrificing performance to some extent. The program used a JavaScript library called SANN (Simple Artificial Neural Networks), which was written from scratch. However, the already existing libraries such as TensorFlow.js and Brain.js could also be used to achieve the same results. The neural network used by the simulation can be seen in fig. 1.
Fig. 1. The neural network used for the simulation.
The project was given a user interface, with visuals, cars and the road, as well as tables for data logging and the ability to export and import the trained model as a JSON file. All values such as vehicle speed ranges, vehicle spawn rates, sensor types, rewards and punishment multipliers, mutation multiplier and so on could be adjusted live and also exported.
The software was run at a high simulation speed, inconsistencies and undesirable behaviors were discovered and logged, and then necessary adjustments were made accordingly each time before the simulation was restarted.
The implementation had a lot of similarities with reinforced learning. However, there is a big distinction. The rewards and punishments in this demo contributed towards a fitness score, which determined which model in each generation was more likely to survive to the next. In the next generation the weights were changed through mutation.
3.1. Simulation environment
The simulation environment consisted of a straight road with four lanes. The two on the right represented the direction of travel for the smart car, with other cars spawning on the road driving in the same direction, and the two lanes on the left were for the incoming traffic.
Other cars on the road were intentionally unintelligent. They only drove straight with a static velocity without switching lanes or trying to avoid the protagonist (smart) car. Making other vehicles intelligent to some degree would make it difficult to evaluate if the protagonist car is learning to drive, or if the environment is simply adapting.
Fig. 2. The simulation environment.
Incoming and ongoing traffic spawned on the road with varying speeds at certain adjustable rates. Occasionally and intentionally two vehicles spawned side by side in two lanes, which made it difficult for the smart car to simply stay in one lane. The smart car therefore needed to adapt, learn to switch lanes, accelerate and brake to avoid traffic.
3.2. Sensors and inputs
The smart car was given five sensors (in shape of rectangles, see fig. 03). One long front facing sensor to detect cars directly in front, two sensors on either side of the car to detect cars on other lanes and on each side, and two diagonal sensors on the front-right and front-left sides of the car. Each sensor was given a collision detection functionality to detect other cars and simulate real-world sensors and cameras.
Fig. 3. The car sensors as inputs.
The size of these sensors was adjustable (simulating effective detection range), and during simulation, the rendering transparency of these sensors could be manually set for better visual comprehension. All other vehicles lacked any sensors except a basic linear distance calculation in order to avoid unintended accidents.
3.3. The neural network and machine learning
The neural network chosen for this demo was a dense, fully connected feed-forward artificial neural network. The SANN library also supported convolutional neural networks and back propagation, but that was not how the learning algorithm had been chosen for this particular project. The network itself consisted of four layers with the input having 10 neurons, the hidden layers 14 and 10 neurons each and the output having four neurons.
A generation in this demo was different from an epoch in conventional neural network training. It was defined to be 12 different models competing with each other to maximize the fitness score. When the simulation started 12 distinct models were generated. Once a simulation stage ended due to an accident, the next one began until all twelve were through. Then the best fitting models were chosen to survive to the second generation and new descendants were generated from those models and so on. The following pseudo-code describes in the most basic way the underlying logic:
create population
repeat for each generation:
for every neural network:
reset traffic scenario
while car has not crashed:
read sensors
calculate neural outputs
move car
calculate rewards and penalties
store final fitness
rank networks by fitness
keep the strongest networks
create mutated descendants from them
replace old population
start next generation
3.3.1. The input layer
Just like a real driver would use his senses to make decisions during driving, inputs in this simulation were the necessary raw data for the network. Therefore, their number was not arbitrary. They were chosen and labeled as follows:
Front: The sensor input directly in front.
Left: The sensor input directly on the left.
Right: The sensor input directly on the right.
Top-left: The sensor input on the front-left side.
Top-right: The sensor input on the front-right side.
Speed: The speed of the car.
Position: The horizontal position of the car.
Wrong lane: If the car is on the incoming traffic lane.
Front cars: Number of cars in front.
Proper lane: If the car is on the correct lane.
3.3.2. The output layer
The output layer was directly commanding the car to steer and change velocity. Therefore, only four vectors were needed. Two vectors to steer the car left and right on the road, one vector to accelerate and one to brake or decelerate. The four implemented neurons were:
Steer left.
Steer right.
Accelerate.
Decelerate.
3.3.3. The hidden layers
Mapping the ten input neurons to the four output ones was not linear. In addition, complex mapping was in this case needed, because in a lot of cases some inputs are activated at the same time. For instance, when the front sensor was activated while the right side was clear; or if the front, right and front-right sensors were all activated and the car was driving in the incoming-traffic lane.
The first hidden layer had 14 layers to accommodate these complex scenarios. However, mapping 14 neurons to 4 was a stark transition. In order to ease this mapping, a second hidden layer was added to map the first hidden layer outputs to the output layer of the network.
3.4. Fitness adjustment and passive teaching
After each iteration, the fitness score of the model and the weights were stored and later compared with other models of the same generation. Teaching, therefore, occurred by trying to maximize the fitness score. However, implementing this method of teaching was not without challenges. Due to unexpected behavior, new rules needed to be implemented, and the contribution of each action towards the final score had to be manually adjusted before restarting the whole simulation and training.
4. Project challenges
The browser-based implementation was not challenge-free by any means. Some of these were expected to a certain degree. Machine learning implementation in JavaScript, for instance, imposes limitations in terms of processing power and functionality to the point that it might be viewed more or less as novelty than real simulation. Therefore, the performance limitation can also be viewed as an exerted challenge.
4.1. Overfitting
One of the main problems with this demo was poor generalization under distribution shift or even overfitting. The model tended to learn certain reactions based on a set of conditions, while changing those conditions, even slightly could result in either more accidents or new unexpected behavior. The model became quite efficient in driving at high speeds relative to other vehicles in disperse traffic. In later versions, the same trained model was tested in an updated environment, where its performance was considerably worse than the original environment run.
In simulations where training data such as images are used, data augmentation could help against overfitting.[9] For the model described in this article, after each alteration the model would need to be partially re-trained.
4.2. Unexpected behavior
The most educational part of this simulation was when the model was observing the model discover clever ways to take advantage of the reward and punishment system to optimize the score. These behaviors were not intended by any means and showed that a machine learning model such as the one described, did not really have a notion of driving or what the intended goals really were. The models did not really think or make decisions the way a human would. The model did not understand the intended objective; its goal was to mathematically maximize the fitness score.
A few of the most interesting unintended behaviors, along with their corresponding solutions, are worth mentioning:
Problem 1: The car drove touching the right boundary, bypassing all other cars and the simulation altogether.
Solution 1: Initially the car was rewarded whenever another vehicle was overtaken and deleted from the memory, regardless of its position and lane. This reward was changed to be granted only if the car is first detected by sensors and then evaded.
Problem 2: The car tended towards driving in the middle of the road over the yellow line dividing the road.
Solution 2: Define punishment per units of time for driving over the yellow line.
Problem 3: The car learned to drive in the incoming traffic lanes.
Solution 3: Define punishment per units of time for driving in the opposite lanes.
Problem 4: The car drove dangerously and chose having an accident over evading a car by driving over the lines.
Solution 4: Punishment adjustment for driving over the lines and in opposite lanes, and making the accident punishment greater. That way driving over the line would temporarily be preferable for the purpose of dodging an accident.
Problem 5: The car drove slow and chose staying in traffic deliberately.
Solution 5: If two cars or more were detected by the front sensors over units of time, the car was stuck in traffic and would receive negative points per units of time.
Fig. 4. The car remaining blocked behind traffic.
Table 1 shows some of the rewards and punishments in qualitative terms.
Table 1. A few of the rewards and penalties in qualitative terms.
Action
Score contribution
Action
Score contribution
detect and evade danger
reward
briefly enter opposite lane
small cost
remain blocked behind traffic
accumulating penalty
crash
large penalty
drive between lane lines in correct direction
reward
remain in opposite lane or ride center line
accumulating penalty
detect danger and still crash
large penalty
cross center line briefly
small cost
5. Achieving desired behavior
After letting the simulation run and train for 69 generations, the car showed the desired driving pattern, but not before causing 829 accidents. The car was able to maintain a decent speed, detect and overtake the other cars on the road. It detected the traffic correctly, and used acceleration, braking and steering accordingly.
Although the simulation seemed to be successful, and demonstrated the feasibility of a self-learning smart car without supervised training and by relying on a reward and penalty system, the model was far from perfect. At this stage, the biggest issue with the network was, as already discussed, overfitting. Changing the simulation environment would lead to new crashes.
Video 1. Desirable driving after generation 69.
6. Increasing the complexity
In order to confirm the poor generalization under distribution shift or possibly overfitting issue, we needed to increase the complexity of the simulation by adding new scenarios and assets. For this purpose the training was switched off and the following modifications were made:
Two new vehicles were added to the simulation: A truck and a tanker, which have larger hit-boxes, were coded in to see how the car does against larger vehicles.
Lane switchers were added to the simulation: Sometimes a small vehicle appeared behind another, and if the lane to its left was free, it would attempt to overtake.
Construction sites were added to the simulation: Very rarely a part of one lane would be blocked without signs and warnings due to construction.
Fig. 5. Added complexity.
As expected, the already trained model did poorly in these new conditions. The same model that had been navigating the road effortlessly to this point had difficulties getting around trucks, and needed retraining. Every time two trucks spawned side by side on the road the car crashed without exception.
The construction introduced a new challenge for the car, for which it had not been trained. The network had been trained to evade moving cars of the same size and shape. This time the car needed to get by an object, which was stationary relative to the road. It underestimated the braking time and drove into the hit-box over 80% of the time.
The only alteration that did not have a big impact on the model performance was the addition of the lane switching cars. It could still detect them and brake early enough.
Conclusion and future work
Training a self-driving vehicle in a simplified simulation environment through a neuroevolution algorithm with fully connected artificial neural networks based on a reward and penalty system to maximize a fitness score was shown to be feasible. Although the car could navigate the simulation obstacles after 69 generations effortlessly and accident free, the path to reaching that goal was full of educational challenges.
The neural network constantly found new unintended ways to take advantage of the reward system and maximize its score. Those behaviors could not be considered examples of responsible driving, thus the simulation rules needed to be amended constantly.
The final trained model was found to be overfit to the training environment and did poorly in new simulation conditions. Those conditions were the introduction of bigger vehicles and still obstacles for which the model was not trained. The model still performed reasonably well against lane switching cars, even though this scenario was added only after training.
This demonstration, although successful within its limits, needed to be expanded. The self-driving car lacked back sensors and the simulation would require a more complex navigation other than a straight road. The next step from the mentioned simulation would be to introduce other types of roads, obstacles, intersections, curves and way-point-based navigation.
The neural network chosen for this demonstration will certainly not do in a much more complex environment. With the increment of inputs and outputs, the model complexity would also increase, and the problem would become one that would require other solutions such as deep learning. In those scenarios performance would matter; therefore other simulation platforms should be sought accordingly.
References
[1] Singh, S. (2015). Critical reasons for crashes investigated in the national motor vehicle crash causation survey (No. DOT HS 812 115).
[3] Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., ... & Zieba, K. (2016). End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316.
[4] Suo, S., Regalado, S., Casas, S., & Urtasun, R. (2021, June). Trafficsim: Learning to simulate realistic multi-agent behaviors. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 10395-10404). IEEE.
[5] Khawaldeh, B., Mora, A. M., & Faris, H. (2024). Optimizing Steering Angle Prediction in Self-Driving Vehicles Using Evolutionary Convolutional Neural Networks. AI, 5(4), 2147-2169.
[6] Sung, J., Kim, K., & Jo, K. (2025, July). Reinforcement Learning Autonomous Driving via Reward Function Design in Simulation. In 2025 17th International Conference on Human System Interaction (HSI) (pp. 1-4). IEEE.
[7] Lade, S., Shrivastav, P., Waghmare, S., Hon, S., Waghmode, S., & Teli, S. (2021, March). Simulation of self driving car using deep learning. In 2021 international conference on emerging smart computing and informatics (ESCI) (pp. 175-180). IEEE.
[8] Nguyen, T. T. H. T., Dao, T. T., Ngo, T. B., & Phi, V. A. (2024). Self-driving car navigation with single-beam LiDAR and neural networks using javascript. IEEE Access, 12, 190203-190219.
[9] Odey, N., & Marhoon, A. (2023). Feature Deep Learning Extraction Approach for Object Detection in Self-Driving Cars. Iraqi Journal for Electrical And Electronic Engineering, 19(2), 62-69.
