Did you know that every political leader on stage and on live television, could be totally generated by a computer? Through the use of Computer Vision and Pattern Recognition (CVPR), real-time facial expressions can be manipulated by anyone. Companies across the globe are rushing to get their hands on this technology because no longer does an individual have to read a script in front of a camera, or use a teleprompter. It can all be done by a computer generated image, without the person even being in front of the camera.
Real-time facial expression swapping took off in 2015. Now, in 2016, CVPR just changed the game when it comes to live television. Real-time facial deformation technology is not new, but the use of such tech is expanding drastically. Combined with audio manipulation everything can be altered in real time.
RELATED: There is Not Much Time Left The BRAIN of Destruction is Taking Shape!
“We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.”
The Technology, called Face2Face, is set to radically change the stage of live television. The technology is being developed by The University of Erlangen-Nuremberg, Max Planck Institute for Informatics, and Stanford University.
How does this change the game? Tell us what you think in the comments below!
Works Cited
Matthias Nießner. “ Face2Face: Real-time Face Capture and Reenactment of RGB Videos.” Matthias Nießner. . (2016): . . http://www.graphics.stanford.edu/~niessner/thies2016face.html