| abstract |
The invention provides a system and method for real-time positioning and map construction with semantic perception. An image acquisition and preprocessing module acquires RGB images collected by a camera and performs preprocessing to obtain a preprocessing map; the semantic segmentation module performs two steps on the preprocessing map. 3D semantic segmentation to obtain a 2D semantic segmentation map; the depth estimation module obtains the depth information of the preprocessing map, and obtains the depth prediction map of the preprocessing map; the camera pose estimation module performs the camera position estimation according to the input preprocessing map and its depth prediction map. pose estimation to obtain the camera pose matrix; the 3D dense point cloud reconstruction module reconstructs the 3D scene dense point cloud for the current frame preprocessing image and its depth prediction map and the current frame camera pose matrix, and maps the 2D semantic segmentation map On the dense point cloud of the 3D scene, the reconstruction of the 3D scene with semantic information is completed. The invention has accuracy and robustness, and realizes the understanding and reconstruction of the scene only by the monocular camera. |