I saw many takes on Tesla AI Day 2022 that fell into a couple of main buckets. Those were mostly about the robot, Optimus, and we’ve published articles covering that at length. However, I stumbled across some critiques of the FSD Beta portion of the presentation that I found surprising and interesting. These mostly came from someone named Chris whose Twitter bio reads, “B.S Computer Sci, Senior Software Engineer, Deep Learning Researcher, I’m all about self driving cars.” I scrolled through his tweets a bit further and discovered more bits of interesting information that I hadn’t seen before.
One of the tweets that caught my attention the most was the last in a 13-tweet thread, but let’s start at the top of that thread. Chris notes, “Continuing my quick overview of #AIDAY, How Tesla FSD Beta drives in 3 not so simple steps. Over the past year, Tesla has made huge changes to their planning architecture. It almost seems like they drew a-lot of influence from @Cruise 2021 Architecture reveal (Under The Hood).” Interesting. The FSD Beta presentation was much more in the weeds than I expected it to be, laying out the detailed ways in which Tesla was teaching its AI to drive. What seemed sort of subtly presented but was clear nonetheless was that Tesla had changed up its approach quite a bit since the last AI Day in 2021. What was not immediately clear was how much Tesla has gone down a path more similar to what Cruise has been doing.
2/ They take the NN outputs (environment/obj/dynamic actors) plus also multi-modal prediction of the actors (moving away from running a copy of FSD on other actors which I criticized them last year for) Look at Waymo’s TNT, MultiPath & MultiPath++ papers https://t.co/BqE47Ax5x2
— Chris (@Christiano92) October 5, 2022
The key there is that rather than using Tesla’s FSD Planner algorithm to predict where others on the street are going to go (if they were using FSD), Tesla has shifted to using different prediction tools for those other actors that are more in line with what Waymo and Cruise use.
The following is a series of tweets that lays out that process in more detail to help understand what is happening in the brief milliseconds Tesla software is determining where the car should drive.
4/ At this point quantity trumps quality, but the car has to narrow down and pick one to execute every 50ms. So They then run a parallel tree search algorithm on these trajectory candidates.
— Chris (@Christiano92) October 5, 2022
6/ Scoring Algorithms: (A) Collison checks (will this maneuver lead to a collision). (B) Comfort checks (is the steering/acc/braking/jerk profile of this trajectory comfortable?), etc.
— Chris (@Christiano92) October 5, 2022
8/ Scoring Algorithms Continued… (D) A neural network trained on bad examples (intervention data). Its job is to determine whether its likely that the human driver would take over. Think about when you disengaged because the car got too close to a pedestrian…
— Chris (@Christiano92) October 5, 2022
10/ The hope is that this network generalizes to be able to figure out if the trajectory is un-comfortable, unsafe, etc. The goal of these checks is to filter out the bad/un-optimal trajectories leaving the good trajectories to pick from.
— Chris (@Christiano92) October 5, 2022
12 / SDC companies like @Waymo, Cruise and many others have been using ML planners (neural network planners) in their deployed driverless stack (no human driver or supervision). So its good that Tesla is following this direction. https://t.co/gg1koicrDA
— Chris (@Christiano92) October 5, 2022
And now we get to the 13th tweet in the thread. In this one, Chris references a clip from a Cruise presentation in 2021 and then a similar clip from Tesla’s AI Day 2022.
13 / Bonus – Another short clip pointing out how Tesla’s architectural concepts and implementation of their driving policy are clearly influenced by Cruise. Like/Retweet this thread if you want more fact/arch based comparison/analysis.https://t.co/FCGDZtHXHE
— Chris (@Christiano92) October 5, 2022
Clearly, these companies aren’t doing everything the same, but they are perhaps following similar paths more than most people acknowledge, and it seems they may have lined up even more in the past year as Tesla’s approach has become more akin to Cruise’s and Waymo’s.
Chris pulled out another comparison recently that was at least as sharp as this one. For those who follow FSD Beta development closely, Chuck Cook has been a super source of videos and information on YouTube. He’s an objective, useful tester who doesn’t add much fluff and is willing to point out where FSD Beta is not doing well. He is particularly well known for repeated attempts at an unprotected left turn across multiple lanes of traffic. Elon Musk and the Tesla team specifically referenced “Chuck’s left turn” in an FSD Beta update in recent months and at the 2022 annual Tesla shareholder meeting. Chris, meanwhile, points out that Argo AI vehicles can handle such turns very well, as demonstrated in a recent Argo AI presentation (that most Tesla fans surely missed). If you look closely, you can see that turn was performed in August 2020. (Yes, August 2020.)
Argo AI’s vs. @chazman Chuck’s Unprotected Left Turn. What’s Different? Working a-couple times with success versus being able to take the turn a million times without fail. That’s the real bar and that’s what @argoai is meeting in Miami, Florida with NO human driver! #FSD https://t.co/VNGueFQyw5 pic.twitter.com/d3371CPN6O
— Chris (@Christiano92) October 2, 2022
Here’s a recent Argo AI tweet with another short video highlighting its self-driving system navigating tough driving scenarios in Miami:
Awesome Autonomy: A lot of activity happens on the streets of #MiamiBeach! Watch Argo Drive, our autonomous system, in action as it navigates a four-way stop & goes around a truck stopped in-lane.
Learn more about how our AV “sees”: https://t.co/odPlTfniZj pic.twitter.com/b0MMSoSu0h
— Argo AI (@argoai) September 29, 2022
Naturally, we all know that Argo AI and Waymo and Tesla use different hardware. Tesla relies solely on cameras for its sensor suite now, whereas others use radar and lidar. Argo AI just recently announced a commercial line of autonomous vehicle products and services:
[embedded content]
The expectation many Tesla fans have is that Tesla’s low-cost hardware (cameras) approach will be more cost efficient, that Tesla DoJo will be able to crunch data in an unmatched way, that Tesla’s overall approach will allow Teslas to drive everywhere while vehicles from companies like Argo AI, Waymo, and Cruise will be much more limited geographically through narrow geofencing.
Nonetheless, sometimes people think Tesla is doing much more original work than it may be doing, and what these other self-driving companies are doing is often underrated and underacknowledged. In fact, in some cases, while it may seem to many of us that Tesla has invented something amazing, it’s actually using another company’s product. Chris highlighted one of these cases as well:
This is just a slightly modified Houdini city generator developed by Epic Games for The Matrix Awaken demo (available for free). Tesla just integrated their auto-labeled HD map. There’s nothing proprietary other than the 3d assets, catching up to where other were years ago. https://t.co/JAw6GBI9B9 pic.twitter.com/fdyN8NXZGx
— Chris (@Christiano92) October 1, 2022
I have to admit that I expected Tesla did more of the work of developing that realistic virtual world, and didn’t realize the possibility to use such city visualizations as a third party. That’s not to say Tesla isn’t using the tech in a novel way and more quickly advancing FSD through that approach. Maybe it is, maybe it isn’t. Though, Chris implies here that Tesla is behind rather than ahead in this regard. Following up on that tweet, he brings up the recollection that Tesla fans trashed the idea of using simulations like this some years ago. Chris adds more perspective here on use of such simulations:
In comparison, here is Cruise explaining their simulation tech that they have had for years. #FSDhttps://t.co/MmuAICoX5w
— Chris (@Christiano92) October 1, 2022
TLDR: Tesla’s new procedural city generation tool is the free Houdini city gen that comes with The Matrix Awaken demo (also free) #AIDay
— Chris (@Christiano92) October 1, 2022
So, now that Tesla is highlighting this as something new, is it a sign that Tesla is years behind? Is the point that Tesla just didn’t need such simulations until now?
I don’t have answers. I don’t have the AI experience to evaluate, and it seems even many experts don’t since AI experts disagree on a variety of important factors. However, what I enjoy learning is where there’s overlap in their approaches, where Tesla adopts things others were using, where others adopt things Tesla was using, and how the experts debate who is winning and who is not. (I also enjoy going through the assumptions and forecasts in my head, privately, where I’m safe from getting dragged by people who either know much more than me or who just don’t agree with my assumptions.)
I’ll just end with a final tweet from Chris. He noted at the end of September, “Mobileye’s hands-free Door to Door system on the Zeekr 001, that works on any China road that has REM with driver monitoring safety assurance is in the final validation stages and will be released to early beta testers momentarily (most likely as version ‘4.0’).”
Mobileye’s hands-free Door to Door system on the Zeekr 001, that works on any China road that has REM with driver monitoring safety assurance is in the final validation stages and will be released to early beta testers momentarily (most likely as version ‘4.0’) #FSD #ADAS pic.twitter.com/5rSfv2CY1O
— Chris (@Christiano92) September 29, 2022
Tesla’s FSD Beta can basically go door to door (Tesla needs to work on adding navigation-guided parking to the stack), the first system to do so. As we know, though, it requires a lot of human supervision and can make all sorts of mistakes. When the Zeekr 001 comes out with these features, how well will they work? How will the performance be similar to what you get with Tesla FSD? How far away from a completely driverless option will Mobileye & Zeekr be?
We’ll see.
Appreciate CleanTechnica’s originality and cleantech news coverage? Consider becoming a CleanTechnica Member, Supporter, Technician, or Ambassador — or a patron on Patreon.
Don’t want to miss a cleantech story? Sign up for daily news updates from CleanTechnica on email. Or follow us on Google News!
Have a tip for CleanTechnica, want to advertise, or want to suggest a guest for our CleanTech Talk podcast? Contact us here.
Advertisement