Master's Thesis, Tampere University (Finland), 2026

Evaluating Rock Position Estimation Pipeline Using 3D Sensors for Excavation Task

Saku Hakamäki saku.hakamaki@tuni.fi Autonomous Mobile Machines Group, Tampere University

Abstract

For the rock-picking task of a real excavator, this work presents an evaluation of a rock 3D position estimation pipeline using 3D sensors. The proposed solution is based on sensor-fusion between camera and LiDAR sensors, and utilizes a 2D binary mask from an object segmentation model to filter the 3D point cloud. Two different segmentation models are evaluated for this purpose: SAM 2 and a finetuned YOLO26. The evaluation is performed on two different computer systems. The proposed 3D position estimate is compared against a ground truth 3D position representation obtained from the FoundationPose model. Based on the findings, the proposed solution yields a median 3D position estimate approximately 0.20–0.30 m from the ground truth position.

Experiments & Results

Model Overview & Experiment Showcase

Segmentation Model Comparison

Qualitative overview of the SAM 2 (top) and YOLO26 (bottom) segmentation models over 3 experiments. The model input RGB images are on the left, whereas the output binary masks are in the middle. The images on the right show the mask contours overlaid on the RGB images.

Frame-Level Comparison

Drag the slider to compare RGB the input image to the model output. The toggle button below allows to switch the comparison between the binary mask and the mask contour.

SAM 2 (Tiny)RGB Binary Mask

Finetuned YOLO26 (n)RGB Binary Mask

Per-Experiment Breakdown

Proposed solution with the SAM 2 model on the desktop system: estimated 3D position represented with the blue-and-purple sphere. No faults were detected during the experiment, and the 3D position estimate follows the ground truth throughout the experiment.

3D position

Ground truth (dashed line) and estimated (solid line) 3D positions in the base coordinate frame.X-axis in red, Y-axis in green and Z-axis in blue.

Acknowledgements

The thesis work has been completed in the Autonomous Mobile Machines Group , at Tampere University. The thesis work has been funded by the Horizon Europe Project XSCAVE under Grant 101189836. Thanks to Novatron Oy for providing access to the excavator and the data interface.

Citation

BibTeX

@mastersthesis{hakamaki2026,
  author    = {Saku Hakamäki},
  title     = {Evaluating Rock Position Estimation Pipeline Using 3D Sensors for Excavation Task},
  school    = {Tampere University},
  year      = {2026},
  url       = {https://urn.fi/URN:NBN:fi:tuni-202604274316},
}