Drones Help Small Team Tackle Big Campus Digitization Project

Skand’s application of Bentley Systems reality capture technology provides RMIT University with a new view of its building facades and roofs.

At Bentley Systems’ annual Year in Infrastructure conference, the infrastructure solutions developer honors customers that have made innovative use of its software products in a variety of application areas. At this year’s event, the award for Advancements in Reality Modeling went to Skand, a small Melbourne-based startup, for its digitization of the Brunswick campus of RMIT University.

Skand specializes in the inspection of building roofs and facades, using drones to collect visual or thermal information about their condition and heat output. Skand then creates a 3D model of each building using Bentley Systems ContextCapture and applies artificial intelligence (AI) to analyze the collected images and find defects. Users can access the results through a centralized, cloud-based viewing platform, and generate reports about the different types of defects, their relative importance, and specific location.

RMIT approached the company with the goal of creating a digital version of its campus. Skand cofounder Brett Chilton explained: “It was really important to [the university representatives] because we’d worked on a previous iteration of the project in a 2D orthophoto representation, and sitting down with them, they said … you wouldn’t [believe] how many times we were able to use even the 2D orthophoto to share with contractors and people that wanted to get on their roofs and on their buildings, and how powerful it was for them to have that information without having to jump in a car or on a bus, on a train, to travel to the site to do site [reconnaissance] and actually view the campus.”


At Bentley Systems Year in Infrastructure 2018, Brett Chilton explains how Skand creates 3D models of buildings from drone-collected imagery.
 

RMIT’s additional goals for the project included a prioritization of defects found during the assessment of the campus, and accessibility for a wide variety of users. “It needed to be intuitive enough and easy enough … that it could by used by the very green, straight-out-of-high-school building managers all the way through to bean counters, CFOs, and indeed, engineers,” Chilton said.

Finding Flaws without Endangering Workers

The Brunswick project comprised six buildings, 65,000 square meters, and 2,044 images collected by RGB white light and thermal sensors. Although the campus is the smallest in RMIT’s network, size is not the only complicating factor in a project of this type. “The buildings we were working with were quite difficult in that there were various sizes, various levels, and various types of build and building materials, which presented all sorts of problems for us from the modeling side of things, but also the analysis using AI,” Chilton observed.

Australian legal restrictions that limit how close drones can be flown to people or assets were another complicating factor. The drone company Skand collaborated with flew overlapping grid patterns over the campus at three different heights. The first two collected images at two different angles — 60 and 45 degrees — to create models in ContextCapture. “Those two capture lines there are perfect for creating what Bentley wants for the model, but they’re terrible for the inspection analysis piece, because you’re too far away and you’re too remote … the resolution of the imagery is too low.” For this purpose, a third pass was flown as low as legally allowed; “typically ten to fifteen meters off the roof or off the façade,” Chilton noted.

Sixteen types of defects were identified, and classified into three priority levels, depending on the urgency of the problem and the danger it could pose to the building or its users. In addition, RMIT requested the creation of workflows that would suggest next steps for each defect identified. Moss growing in a rain gutter, for example, “would probably be a Priority 3: It’s not going to kill anyone, it’s not going to hurt the building … it would be a ‘Clean or remove next time you’re on the roof,’” Chilton explained.

An Identification Assist from AI

Examining imagery and determining what is and isn’t a problem is a task shared by Skand’s human and computer collaborators. “On a previous project, we had fed 250,000 photos of roofs into the AI platform, and over that data set, we had six categories that lit up where we were actually predicting the possibility of that being a particular defect [better] than a team that we had trained to do inspection. But it’s 2018 and it’s really early days … we have trained the data to find these things, but it still needs the humans’ touch to tell us what level or to what degree that defect is.”

The project yielded a 3D model that the customer can view in Skand’s online platform, rotating and zooming to examine details. The defects, which have been automatically marked by the AI with colored polygons, can be toggled on and off by type; users can also change the metadata associated with defects. “This [view] is pulling from the database live, as we update it,” Chilton said. “It’s very basic, and yet powerful.”

A timeline is also integrated in the interface. “Although most of our clients are in their first or an early scan, the idea is that we’ll be looking at a mesh-to-mesh comparison over time, which is like the holy grail. We’re not there yet, but we’re certainly looking toward that now,” Chilton commented.

Ultimately, the drone-based inspection process saved RMIT $70,000, Chilton said, and reduced risk to workers. Previously, a typical inspection of the campus would take 20 human days to complete, with the participants “either on ropes, or a boots-on-rooftop scenario, and an engineer onsite and an engineer offsite doing the analysis,” he explained. The human involvement in Skand’s version of the inspection — including flying drones and adding metadata to defects — totaled just seven days. And as time goes by, the process will become even more automated, Chilton predicted.