InnoVision Directorate Develops 3D Thermal Site Models30 May, 2012 By: Nathaniel Ovans
The National Geospatial-Intelligence Agency's research and development arm fuses thermal imagery with LIDAR data to provide a new view of buildings and scenery.
When National Geospatial-Intelligence Agency (NGA) director Letitia A. Long announced that she wanted to "fundamentally change the user's experience by putting the power of GEOINT in the hands of the user," Todd Johanesen — the NGA senior scientist for photogrammetry — went to work.
Johanesen, a recipient of funding through the Director of National Intelligence (DNI) Tech Fellow program in 2010, developed a concept to build and evaluate new methods of fusing thermal infrared (IR) imagery with other geospatial intelligence (GEOINT) data sources to provide improved GEOINT data. He developed a 3-D site model that he could register thermal data to using traditional photogrammetry, transforming multiple datasets into one coordinate system. (Photogrammetry is the science of obtaining information from images using measurements and interpretation.)
The 3D Thermal Site Model (3D TSM) is a highly accurate 3D model of a given facility or scene that integrates spatial, engineering schematic or blueprint, and relative temperature information into a single, precise GEOINT product. The operational and intelligence uses for the 3D TSM are boundless. Accurately combining the different sources of information could help analysts perform infrastructure analysis for critical nodes, aid in planning and rehearsing missions, or even support personnel recovery, said Johanesen.
"One example is the ability to rapidly and accurately pinpoint a sniper while pinned in the sniper's line of fire," said Johanesen. "In this scenario, troops employ a simple thermal imager, pinpoint the sniper's thermal signature, register that to a reference light detection and ranging (LIDAR) point cloud, and deliver an accurate description of the sniper's 3D geolocation."
LIDAR is one of the many phenomenologies that NGA uses to create 3D ground terrain maps. NGA's Photogrammtery and Image Science Division (IBP) led research into the acquisition, fusion, and visualization of data to support 3D TSM generation. The TSMs combine known information from a photogrammetric site model with thermal data captured from handheld, airborne, and/or spaceborne perspectives. The key is establishing metadata elements of all sources, to include the thermal imaging system, to integrate additional data points such as temperature.
This figure depicts a registered thermal map overlay (colored area) on a LIDAR point cloud.
To investigate the potential, IBP acquired a highly accurate (photogrammetric) commercial off-the-shelf handheld long-wave IR camera and developed a sensor model for it. This sensor model supports current research on fusion of thermal data from this camera with LIDAR point cloud data. Scientists manually register datasets today, but IBP is researching automated methods of registering the datasets to support a more timely and automatic capability. This combination yields thermal data with predictive estimates of the data's absolute accuracy relative to actual position on the Earth and facilitates more accurate fusion with other data sources.
IBP scientists tested the data in the field, using an IR camera, an optical camera, and a terrestrial LIDAR system to image buildings of different materials and shapes, along with scenery. "The camera was easy to use compared to other similar sensors and is portable, it has UAV (unmanned aerial vehicle) application, and the application of using the camera in an unattended mode looked promising," said Robert Lund, former Southern Command NGA Support Team deputy director.
Using the collected data, IBP was able to register a building's LIDAR point cloud with its thermal profile, creating a 3D TSM. IBP was also able to characterize an efficient collection system for gathering thermal data and incorporating it into a 3D TSM, as well as create a prototype software tool for 3D TSM creation and accuracy assessment. The prototype allowed IBP to make an initial assessment on the value of 3D TSMs, which became apparent when the model highlighted and georeferenced unseen building attributes.
This figure depicts a point on a thermal image (left) with its correspondence point and its accuracy noted on the 3D TSM (right).
IBP continues to explore the potential use of IR applications and the desirable attributes of the 3D TSMs used for reference data. The current focus is to broaden community awareness, increase collaboration, and conduct additional research. A key goal is to calibrate current operational IR sensors and develop sensor models so that scientists might use operational IR data to generate 3D Thermal Point Clouds to support time-phased analysis like change detection. Additionally the team will continue to explore the structure and content of photogrammetry-based 3D TSMs to increase the value of the GEOINT reference products used to fuel thermal fusion.
Editor's Note: This article was originally published in the March/April 2012 issue of the National Geospatial-Intelligence Agency’s (NGA) Pathfinder magazine.
In her easy-to-follow, friendly style, long-time Cadalyst contributing editor and Autodesk Technical Evangelist Lynn Allen guides you through a new feature or time-saving trick in every episode of her popular AutoCAD video tips. Subscribe to the free Cadalyst Video Picks newsletter and we'll notify you every time a new video tip is published. All exclusively from Cadalyst!