3D Markerless Reconstruction of Flying Bats and Membrane Simulation

Presenter: Yihao Hu, Mechanical Engineering

Authors: Y. Hu, C. Nnoka, J. Tan, . R. Müller

Abstract: Bats are capable of remarkably dexterous flight maneuvers that rely heavily on highly articulated hand skeletons and malleable wing membranes. To understand the underlying mechanisms, large amounts of detailed data on bat flight kinematics are required. In this project, an automated pipeline was developed for reconstructing a flying bat using articulated template and images from high-speed cameras only. The pipeline consists of two functionalities: kinematics reconstruction and membrane motion reconstruction. Kinematics reconstruction uses joint rotation matrices to articulate the digital template, and those rotation matrices are optimized so that the rendered silhouette images of the deformed mesh fit those recorded by cameras.  Membrane motion reconstruction requires reconstructed kinematics, along with a mesh template where the membrane area is modeled using a spring-mass network, interacting with a physics simulator to obtain the realistic motion of the membrane.  The physical parameter of the membrane is optimized using the model rendered and recorded silhouette images. Intersection of union (IOU) and 3D Euclidean distance is used for qualitative evaluation, and the total number of reconstructions is about 30,000.