|
|
Loading ...
|
|
|
|
|
pages views since
05/19/2016 : 154693
· Members : 7
· News : 884
· Downloads : 0
· Links : 0
|
|
|
|
|
Witnessing the Vanishing Act: Diminished Reality in Live Video.
|
|
|
|
Posted by Okachinepa on 04/10/2025 @
Courtesy of SynEvol
Credit: Graz University of Technology
Scientists at the Institute of Visual Computing have achieved the ability to eliminate objects from real-time recordings of 3D environments without any lag, even as the camera moves.
The integration of digital objects into a real environment is widely known as augmented reality (AR) today. Its equivalent, diminished reality (DR), is not as widely recognized: DR allows for the removal of objects from a scene and fills the created gap in a believable manner.
To accomplish this in a 3D environment, earlier techniques demanded significant computational resources and could only produce the modified scene with a time lag and occasionally inaccurate depictions.
A group directed by Dieter Schmalstieg, Shohei Mori, and Denis Kalkofen from the Visual Computing Institute at Graz University of Technology (TU Graz), alongside Hideo Saito's team at Keio University in Japan, has successfully achieved the real-time disappearance of 3D objects through diminished reality by integrating multiple technologies. This enables a space to be observed in real-time via a camera in its modified state.
"In simple terms, Mori describes diminished reality as a form of Photoshop specifically for three-dimensional scenes." "Some individuals may initially consider how they can utilize it to eliminate unwanted elements from holiday footage." It is difficult but could be achievable. There are additional domains where DR applications prove to be highly beneficial.
This creates new opportunities in the industry for simulating failures in training datasets for self-driving cars or for the effective pre-visualization (previs) of movie scenes at actual filming sites that appear entirely different from what is intended during location scouting.
DR could likewise benefit the medical sector, for instance by eliminating distractions from the view of recording cameras during surgeries to deliver uninterrupted learning resources for students.
The innovative technology, named InpaintFusion, begins with 2D inpainting and subsequently removes objects from a 3D scene. User inputs on the 2D display are projected into a 3D environment to enable the specification of regions to be eliminated. A keyframe acts as the initial reference, and a believable backdrop is formed by gathering and blending corresponding pixels from the area surrounding the object that is to be eliminated.
To accomplish this in 3D, the color details and depth information of the captured scene are enhanced simultaneously, ensuring that the outcome appears realistic even with changes in the camera angle. The initiative has accomplished the implementation of 3D inpainting on volumetric 3D image data (multi-layer images), resulting in improved quality in the perception of object distance.
 .
Courtesy of SynEvol
Credit: Graz University of Technology
The greatest challenge was enabling the processing steps to occur in real time. The researchers employed two distinct technologies for this purpose: rapid patch matching and multithreading. Through patch matching, the surrounding pixels that are most visually prominent for the area needing to be filled are randomly searched to gather and combine the matching pixels. They don't need to be the absolute finest, but generally, they offer reliable filler content without the need to process millions of pixels.
Multithreading takes advantage of computer processors to run multiple processes at the same time on each processor core. In the research team’s reduced reality approach, the resource-heavy 3D inpainting process operates in the background as the main thread manages the visualization. This enables users to view the final outcome immediately and navigate the room without the eliminated object reappearing in the picture.
Mori states, "The real-time display of reduced reality represents a significant advancement for this technology." "The objective now is to create the necessary toolkits to ensure that these opportunities can be offered to a broader audience." Another aim is to rapidly and effectively create 3D models from a limited number of separate images. This would provide diminished reality with an extra layer.
|
|
|
