![]() We also added more routing paths to make the car handle these new conditions. We started creating more complex situations for the car, such as adding multiple intersections to the tracks. (Left) square shape track, (Right) Curvy trackĪfter fixing the imbalanced dataset, the car began to correctly navigate corners. To fix this imbalance data issue, we added various shapes of curves. Later, we found out that we only trained the model using mostly straight tracks. The car ran on the tracks for a few seconds and then just went off the track for various reasons.Įarly versions of the toy car running off the track/td> Training the ML model using only the simulator doesn’t mean it will actually work in the real-world situation, at least not the first try. Output from the first Neural Network layer We added these variables to the scene: random HDRI sphere ( with different rotation and exposure values), random brightness and color, and random cars. Since we do all data collection within the simulator, we need to create various environments in the scene because we want our model to be able to handle different lighting, background environment and other noises. In this simulator, we collect image data and steering angle every 50ms.ĭata Augmentation with various environments Multiple waypoints on the track in the simulatorīy setting multiple waypoints on the track, the car bot is able to drive to different locations and also collects data for us. We chose Unity and this simulator project from Udacity for lane-keeping data collection. The problem is we didn’t have a car or a track to use at the time. Model = Model(inputs=net_in, outputs=net_out) Data CollectionBefore we are able to use this model, we need to find a way to collect the image data from the car to train. Model Architecture net_in = Input(shape = (80, 120, 3)) After experimenting a bit more, we followed a similar model architecture to this paper. We improved this by adding an LSTM and using multiple previous frames. As a baseline, we used a CNN to detect the traffic lines in each frame and adjust the steering wheel every frame, which works fine. Lane-keepingWe explored a variety of models for Lane-keeping. If you’re interested in technical details, the remainder of this article describes the major components of the car, and our journey building it. Not only that, in our case, the Pixel 4 also controls the motors and other electronic components via USB-C, so the car can stop when it detects other cars or turn at a right intersection when it needs to. How it worksUsing the front camera on a mobile device, we perform lane-keeping, localization and object detection right on the device in real-time. ![]() ![]() As you may notice from the gifs below, Pixelopolis has multilingual support built-in as well. The car will navigate to the destination, and during the journey, the app shows real-time streaming video from the Car - this allows the user to see what the car sees and detects. Users can interact with Pixelopolis via a “station” (an app running on a phone), where they can select the destination the car will drive to. If you can, doing it on-device is much faster. Processing video and detecting objects are much more difficult using Cloud-based methods - due to latency. In order to sense lanes, avoid collisions and read traffic signs, the phone uses machine learning running on the Pixel Neural Core, which contains a version of an Edge TPU.Īn edge computing implementation is a good option to make projects like this possible. Each car is outfitted with its own Pixel phone, which used its camera to detect and understand signals from the world around it. PixelopolisPixelopolis is an interactive installation that showcases self-driving miniature cars powered by TensorFlow Lite. I wish we had the opportunity to meet in person, but I hope you find this article interesting nonetheless! In this post, I’d like to share with you a demo we built for (and had planned to show at) Google I/O this year with TensorFlow Lite. Posted by Miguel de Andrés-Clavera, Product Manager, Google PI
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |