Autonomous Robotics System with YOLO + LiDAR

A robotics prototype that combined YOLO vision with LiDAR mapping and obstacle sensing, then used detected objects as inputs to navigation and manipulation experiments.

My role: Robotics and computer-vision developer
Context: Hochschule Trier exchange semester
Reading time: 5 min read

Academic robotics project

Vision and distance sensing solve different problems

YOLO could tell the robot what kind of object appeared in the camera image. LiDAR could describe nearby geometry and obstacles even when visual classification was uncertain. The prototype combined those signals instead of treating either sensor as a complete perception system.

This let the robot connect semantic information from the camera with spatial information from the range sensor. The result was a more useful basis for mapping and movement than either raw stream provided alone.

Objects as navigation waypoints

The path-planning work used detected objects as navigation waypoints while LiDAR data helped the system avoid obstacles and maintain an environmental map. The robot had to translate a detection in image space into a target that made sense in its movement and map coordinates.

We tested the behavior in simulation and real-time experiments. Those tests made calibration and coordinate assumptions visible, especially when detections were correct but the physical path was still poor.

From recognition to a useful grasp point

Recognising an object does not tell a robotic arm where to pick it up. The project also applied computer-vision techniques to locate a practical grasp point for manipulation tasks.

That part of the work connected perception with action. It required a stable image, a useful object region, and geometry that the arm could reach, not just a correct class label.

What the prototype taught me

Robotics exposes every assumption in an AI pipeline. Timing, calibration, coordinate transforms, sensor noise, and actuator limits can turn a good isolated model into a poor system.

The project remained an academic prototype, but it gave me a direct view of how computer vision, sensor fusion, mapping, path planning, and manipulation depend on one another.

Outcomes

01A prototype combining YOLO detections with LiDAR obstacle sensing and mapping.
02Navigation experiments that used recognised objects as waypoints.
03Computer-vision work for identifying grasp points in manipulation tasks.

What I learned

01A correct detection is only the beginning of a robotics decision.
02Sensor fusion becomes useful when coordinate frames, timing, and uncertainty are handled explicitly.
03Simulation helps isolate logic, but real-time tests reveal the calibration and latency problems that matter.