At the Monet group in UIUC we work on the emerging tele-immersive 3D multi-camera room environments. These environments allow us to engage in distributed physical activities such as physical therapy, sport activities, and entertainment.
Such environments need 3D multi-camera setups at the sending side and multi-display setups at the receiving side connected via appropriate network infrastructure. They also pose a large number of challenging research questions.
We are building a cross-layer control and streaming framework over general purpose delivery infrastructure, called TEEVE (Tele-immersive Environment for Everybody). We tackle problems such as capturing, reconstructing and displaying 4D content, and coordination, synchronization and QoS-enabled delivery of tele-immersive 3D visual streams to remote room(s) over Internet 2.
The first experiments of TEEVE and few other next generation architectures are encouraging, as we start to sustain communication of up to 12 3D streams with several frames per second (e.g., TEEVE can reach around 5 frames per second) over Internet 2. However a lot of work remains and a lot of problems need to be solved.
This project is supported by the National Science Foundation and National Center for Supercomputing Applications.