Virtual fixture generation for task planning with complex geometries

Sharp, Andrew Patrick
Journal Title
Journal ISSN
Volume Title

Decontaminating and decommissioning aging nuclear facilities and managing nuclear waste required increased automation to reduce personnel dose. Semi-autonomous behaviors, such as virtual fixtures, aid task execution by managing low-level system resources while operators retain high-level control. Virtual fixtures provide operators with geometric constraints or guidance forces in a robotic manipulator’s workspace. This dissertation advances virtual fixture generation through, the previously unexplored, construction of layers of point cloud based Guidance Virtual Fixtures. Point clouds are used for virtual fixture generation, based on complex surface geometry, to provide more expressive, and therefore useful, environmental representations. Thus, this work builds upon previous point cloud based Forbidden Region Virtual Fixtures to address virtual fixture generation shortcomings outlined in the literature. Task input polygonal mesh checks warn operators if defects are found. Task normal vectors and task parameters are used to calculate point cloud layers at task defined distances from the surface. These layers are interpolated and voxelized to maintain point cloud resolution at increasing distances from the task surface. The layers are combined into a bi-directional graph structure for storage and future use. The graph structure is combined with a Forbidden Region Virtual Fixture to create a Task Virtual Fixture. Task Virtual Fixture generation was evaluated with multiple input types including parametric surfaces, polygonal meshes, and point cloud data. Results demonstrate surface model concavity affects the growth in the number of offset layer vertices as does distance from the task surface. Task Virtual Fixture generation intuitively modifies VF layer resolution at extended task surface distances. Point cloud sensor data demonstrated sensor data input for "open world" scenarios. Two visualization and task execution environments were developed to apply Task Virtual Fixtures to spatially discrete and spatially continuous non-contact tasks. The first interface, spatially discrete, was constructed with the Robot Operating System, RViz, and MoveIt!. This interface displays reachability information to the operator and is called the Manipulator to Task Transform Tool. The second interface allows operators to employ Task Virtual Fixture information in ABB’s RobotStudio for spatially continuous tasks. A small user study was conducted for each interface to demonstrate more expressive Task Virtual Fixtures are still operator interpretable and assist with task execution.