• 3d modeling
• 3D reconstruction
• user-assisted 3D reconstruction

Three-dimensional (3D) reconstruction aims to create a 3D model of an object or a scene from data captured in the real world using depth sensors such as LiDAR or RGB cameras. It has applications in fields such as virtual reality, augmented reality, architectural/mechanical engineering, medical imaging, and so on. Structure from Motion (SfM) emerges as a popular form of 3D reconstruction, due to cost-effectiveness and ease of use. Existing SfM tools struggle with 3D reconstruction for low-resolution, blurred, compressed images, or objects with challenging textures like shiny or transparent materials. In these situations, prior information or users' hints can improve the quality of results. Low-cost LiDAR scanners generally give low-resolution and non-uniformly distributed point clouds, which cause issues in point cloud registration for 3D reconstruction of the scene. This thesis focuses on addressing the 3D reconstruction of such difficult cases. We provide three key contributions in this thesis.

The first and most important contribution is the development of a novel graphical user interface named Movie Reconstruction Laboratory (MoReLab) for interactive 3D modeling and user-assisted 3D reconstruction. MoRelab is an open-source software, and it is available for all non-commercial uses. It allows the user to import a video, extract frames, specify feature correspondences in those frames, and estimate camera parameters and the 3D points corresponding to marked features simultaneously. Then, the user can calibrate and perform measurements. In addition, the user can make use of primitive tools to model surfaces such as complex and curved pipes. This is extremely important to model industrial equipment, which is the main target in our research, and to measure them. The graphical user interface of MoReLab is standard. However, the main innovation of MoReLab lies in the innovative tools that have been introduced in MoReLab. Specifically, MoReLab introduces two novel tools:
(1) Guiding lines tool
This tool is used to guide the user about the location to add feature correspondences. It works on the priciple of epipolar constraints and calculation of Fundamental matrix.
(2) Curved cylinder tool
This tool exploits Bèzier curves to approximate and model curved pipes and cylinders. This thesis presents detailed comparisons of modeling curved pipes using curved cylinder tool of MoReLab and other softwares. Comparisons of these results in Chapter 3 and Chapter 4 show that MoReLab achieves superior results as compared to other state of the art tools.

The second contribution is a user study to evaluate MoReLab against the existing user-assisted 3D reconstruction tools. Results from this user study establish that MoReLab outperforms the existing user-assisted 3D reconstruction tools in terms of visual modeling results and measurement error accuracy. Software evaluation shows that MoReLab is preferred by novices and experts in terms of user-friendliness, easiness, and performance improvements.

Finally, the third contribution is to develop a sequential pipeline for the registration of low-resolution and non-uniformly distributed point clouds. Merging sparse reconstructions is important to enable real-time 3D reconstruction of certain data sources, like for example a Lidar mounted on a drone or a car. The approximation of alignment error seems to fail in some cases to recognize low-quality registrations.

Overall, this thesis addresses the challenging inputs for 3D reconstruction from uncalibrated images or a video. Experiments show that MoReLab surpasses other state of the art tools for interactive 3D modeling and 3D measurements. However, there is still room for further developments and improvements. We have seen that Neural Radiance Fields and Gaussian Splatting have emerged as powerful tools for 3D reconstruction from high-quality images in the past few years. User interactions such as thoses introduced in MoReLab can be useful to calibrate low-quality images and challenging materials and then further obtain high-quality 3D reconstruction results with Neural Radiance Fields and Gaussian Splatting.