Loading...
Adapting Vision–Language–Action Models to New Domains: A Case Study with SimLingo on QCar 2
Files
Loading...
azu_etd_23151_sip1_m.pdf
Adobe PDF, 4.61 MB
Citations
Altmetric:
Author
Affiliation
Issue Date
2026
MeSH Subjects
Advisor
Collections
Other Titles
Citation
Mazmanyan, Garegin. (2026). Adapting Vision–Language–Action Models to New Domains: A Case Study with SimLingo on QCar 2 (Master's thesis, University of Arizona, Tucson, USA).
Rights
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction, presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Permanent link to this item
Embargo
Publisher
Journal
Table of Contents
Abstract
This thesis documents an evidence-based engineering adaptation of a multimodal Vision–Language–Action (VLA) driving policy, SimLingo, originally trained from the CARLA autonomous driving simulator to the Quanser QLabs vehicle control platform controlling a QCar 2 vehicle model. The contribution of this work is a reproducible end-to-end pipeline spanning: (i) expert driving data collection in QLabs, (ii) LoRA-based fine-tuning of an InternVL2-1B backbone using collected expert driving data, and (iii) a real-time inference stack that converts model outputs (waypoint trajectories) into low-level QCar 2 control commands using a simulator-specific PID controller. The adaptation addresses platform mismatches that materially affect closed-loop behavior, including coordinate frame conventions, control-rate differences, camera resolution preprocessing, and speed estimation consistency with the training data. Experimental evaluation on a roundabout scenario with five obstacle placement variants shows that the fine-tuned SimLingo model achieves a 60% overall pass rate (9/15 runs) and a 40% obstacle avoidance pass rate (4/10 runs), successfully detecting and stopping without contact for obstacles in two of five variants. In two additional variants, the model demonstrates clear obstacle detection and deceleration behavior but makes low-speed bumper contact before fully stopping. Compared against a LiDAR-based adaptive cruise control baseline that achieves 100% pass rate on the same test scenarios, the vision-only model shows worse but promising performance. Offline validation shows a 25% reduction in Average Displacement Error (ADE) (from 0.114 to 0.085) over 15 training epochs, converging toward the expert demonstration baseline (ADE=0.087).
Description
Note
Type
text
Electronic Thesis
Electronic Thesis
Series/Report no.
Degree Name
M.S.