Body Analysis: Privacy
Masking Humans using real-time segmentation
Project Information
Privacy concerns from a legal standpoint can be covered with in-app messaging. However, there is still doubt and misunderstanding when it comes to technology: the default position is not to trust what the eyes cannot see (and more)
Company
Advanced Health Intelligence
The visual abstraction of the self-image, which is partially clothed or nude will have a direct effect on what a person thinks is being recorded or seen. Showing them a mask of their body and a different background removes their personal identity from the image being displayed.
The psychological effect is two-fold as the background is no longer visible, and the person is just a shape. The effects are immediate and a person will quickly move onto how ‘cool’ it is to see their shape on the screen moving around in real-time.
Masking a Human
New Challenges
There is still friction around how a person might view themselves even if masked. If someone is unhappy with the size and shape of their body, the effect will be increased. To address this, using a 3D avatar in place of the individual, and reacting real-time to their joints is a possibility.
Real-time Matte Effects using VisionFramework
I remembered a project from WWDC 2016 from Apple that was close to what I had in mind. I modified the code in Xcode and put together a demo that showcased the effect that I was after.
An important aspect of recommending technology to the engineering team is to have some idea of the work involved to reach a stable version, particularly around performance on various devices.
I understood my demo was iOS only and that on slow devices it might lag when using accurate masks, so degrading to fast masks was an option. Typically, we would reach a stable release on the latest devices on the suitable platform and then trickle scheduled work to cover the remaining devices and platforms.
Performance differences in masking
Advantages
There are advantages of using this capture technique outside of privacy alone, such as:
⁍ Full image/video backgrounds
Animated Backgrounds
Unlockable Backgrounds
There are advantages of using this capture technique outside of privacy alone, such as:
⁍ Full image/video backgrounds
⁍ Clean real-time joints – by using colored masks, the joints can be enlarged, create a high contrast, and improve legibility from a distance.
⁍ Reduced visual clutter in determining limbs into outline.
⁍ Alternate capture through fit-frame modeling (no outlines).
Clean Joints