Solving the Naked Guy Problem in Drone Data

Picture this: you're a drone pilot doing contract work for the government, who want you to make orthomosaic maps of water treatment facilities. It’s a lovely summer day.

The flight goes well, and when you get home, you upload the data into your photogrammetry software. It spits out a lovely-looking orthomosaic map, exactly what your client asked for this. Then you notice a strange pink spot, right on top of a low-lying concrete structure.

You zoom in.

It's a very naked guy, who's tanning on a blanket. You somehow didn't notice him when you were flying the drone.

You have been confronted with the Naked Guy Problem in drone data collection. Allow me to explain.

While our drones may give us the kind of god's-eye-view that our ancient ancestors could only dream about, most of the time, we don't want them to see everything. As drone professionals, we instead want our aircraft to practice a certain amount of discrete selective blindness, like the kind we practice when we see one of our esteemed colleagues picking their nose on a Zoom call.

When we fly drones for professional purposes, we're usually doing it because we want to look at a few specific elements. Everything else in the picture is just background noise, incidental data that our drone’s camera happens to pick up along the way.

From an analysis perspective, it's enough of a challenge to pick out the stuff you want to see in a drone photo from the stuff you don't want to see. But that's not where the problem ends. That’s because that background information in a drone photo is a potential privacy (and PR) nightmare. It’s not just irrelevant. It’s actively dangerous.

While the drone pilot couldn’t care less about capturing saucy photos of a naked guy sunning himself right on top of a water treatment building, the naked guy doesn't know that.  All he knows is that there was a drone hovering over his head, piloted by unknown parties with unknown intentions. It’s quite likely that he doesn’t want photos of himself in the nude on the Internet, or lurking within state government archives.

The naked guy may be so disturbed by this prospect that he starts looking into what parties might be flying drones over his favorite water-treatment facility sunning spot (which he’s legally permitted to use in your area), which will lead him to finding out about the government drone-mapping project. And then, perhaps, to a lawyer.

And even if the naked guy actually is someone who loves the idea of a government drone memorializing his bits for posterity, that’s probably not what the government agency you’re flying the drone for wants. They don’t want to be hosting pictures of random naked guys on their servers, either.

This, then, summarizes the Naked Guy Problem.

What do you do if your drone picks up data that you don’t want, the person in the picture doesn’t want, and that your client doesn’t want? Especially if the Naked Data happens to intersect directly with data you actually need?

In the drone world, we usually refer to imagery like “photographs of naked guys you didn’t meant to take” as PII, or personally identifiable information. PII is a technical term that has different formal and legal definitions depending on who you're asking. The US Department of Labor calls it “any representation of information that permits the identity of an individual to whom the information applies to be reasonably inferred by either direct or indirect means.” Meanwhile, the European Union defines “personal data” as “any information that relates to an identified or identifiable living individual.”

Most people aren't aware of these formal definitions. Thus, in the spatial data worlds I tend to work in, "PII" has become something of a stand-in for "any information that someone you're flying a drone over might get irate and threaten legal action over, or that might land you in hot water with the government."

This informal definition includes an exceedingly wide range of potentially problematic information, from pictures of people's backyards to pictures of people's cars (especially if the drone captures a pattern of moment), to zoomed-in shots of license plates, the top of people's heads, and much, much more. Such as, you know, naked guys, who happen to be lying down in such a way that they are very identifiable indeed from a low-altitude drone mapping flight.

So what are our options, as drone pilots, when we want to make sure we're picking up as a little potentially-problematic PII in a drone flight as humanly possible? How do we avoid this kind of toxic-asset data that we never wanted anything to do with in the first place?

The first line of defense: flight planning. If you’re flying in a populated area or anywhere else with even slightly elevated risk of naked-guy appearances, you should make ensure your drone is capturing photos or video of the absolute minimum amount of ground required to achieve your goals. Remember that “collecting more data than you need because you can” is a much more dangerous practice than most people think it is.

Pilots should make every effort to identify naked guys while they're flying the drone, or preferably, before they launch. Visual observers should be deployed on initial naked-guy sweeps before takeoffs. Mobile device or controller screens should be monitored for naked-guy appearances.

Unfortunately, flight planning is imperfect. The drone's viewfinder is very often going to catch some areas that the pilot didn’t want to catch. The VO will fail to spot the naked guy, and the pilot may flick their eyes away from the screen at the exact wrong moment. Naked guys will, sooner or later, creep into the proceedings.

What if these efforts do fail, and you discover a naked guy in your drone photos, or in a map you’ve already processed?

You’ve done the first part of your job right. That is, you reviewed the imagery in the first place before you released it into the wild.  Far as I can tell, very few (or no?) organizations that collect a lot of drone data have a truly formal PII review process. In my experience, whoever ends up processing the UAS data into an orthomosaic map just zooms in close-ish and looks for anything suspect. And that’s if they remember to do it - which they often don’t, especially if review isn’t any kind of organizational priority.

While we probably, technically, should be reviewing every single drone photo individually before start processing a map, just about nobody does this in practice. In my experience, the vast majority of Naked Guy problems are detected after photogrammetry software has already converted the photos into a map.

You can try leaving out the individual naked-guy UAS photos and reprocessing the map without them. However, this isn’t a great solution. You’ll be sacrificing quality and accuracy for a wonky looking final product. You’ll also be missing those part of the map that contain both information you need and the naked guy.

Some cloud-based photogrammetry software tools, like DroneDeploy and Esri’s Site Scan, now offer ways to crop processed drone orthomosaics. This is a nice solution if your naked guy problem (or that backyard where they’re having a pool party) happens to be on the outskirts of your image.

In truth, using these cropping tools should be a standard part of your Naked Guy avoidance workflow: make sure you crop everything you collect to the exact boundaries of what you need before you export it.  (Oh, and if you’re working for the government, you should really review what your organizational policies are around retaining and deleting drone data).

You could also fire up Photoshop and carefully crop the naked guy out of your raster, although this will leave a suspiciously guy-shaped hole in the image.

The best solution I’ve found? Pix4Dmappers’s Image Annotation feature, which lets you remove objects from an orthomosaic. While I haven’t used it to remove actual naked guys, I have used it to remove moving cars and other objects from orthomosaics. The results look natural. You’ll avoid embarrassing questions.

"That’s nice, but Pix4Dmapper is expensive and doing this kind of thing manually seems like a real pain in the ass," you might say. “Surely, in these days of miracle and wonder, there exists an automatic means by which I could review my drone data for the presence of unauthorized naked guys.”

No, not really! I'm afraid that you can't just buy a subscription to Naked Guy Detector 2021: Drone Edition for your institution or business.

While there are automated methods for redacting other kinds of footage and photos, like police body cam video, no such thing really exists for drone data at the moment. Although a few companies sort-of-claim that they can redact drone data for PII, I haven’t seen the evidence.

Automatically identifying PII from the air is a challenging problem, especially because the objects we’re most interested in tend to be really small and varied. This issue is compounded by the fact that drone data tends to be shot at exceedingly variable altitudes and angles, at different times of day, and with very different cameras and sensors. (I’ve written about the technical challenges facing drone facial recognition before).

I don’t think it’d be impossible for someone to build a half-decent “this thing might be PII” detection tool for drone photographs and orthomosaics. I’d certainly use it if it existed. (I’ve even contemplated building such a thing myself).

Because the technical issues are challenging but not insurmountable, I find it rather interesting that some version of “Naked Guy Detector 2021: Drone Edition” doesn’t exist yet.

Is this because the drone industry genuinely doesn’t have as many Naked Guy problems as I think it does? Or is it because the drone industry doesn’t think the Naked Guy problem is that big a deal? Or, perhaps, is it because we think that avoiding the Naked Guy problem will make it go away?

Whatever the reason, the drone industry will need to hash out better and more standardized methods for reviewing drone data for PII and redacting it in the near future.

It’s simple math. If we expect that there will be a lot more drone flights in the future, we can expect a lot more Naked Guy-type information to inadvertently pop up in the results. And as drones become more commonplace tools for government data collection, more drone data will be made accessible to the general public - creating an urgent need for clear, formal mechanisms for reviewing data and scrubbing it of problematic information before everyone can see it.

All this leads to another question, which I’ll address in my next post.

Do we actually understand what kind of private, sensitive information can be picked-up-on in videos and photos from small drones? How often are we actually detecting naked guys with drone imagery, and how likely is it that someone will identify the naked guy in question? What happens to naked guys when they do get identified from aerial data? Or are we almost all making wild guesses rooted in almost no actual scientific evidence about pretty much all of this PII and drone data stuff?