Author: Bart Kroon

The verification test report of MIV [1] was published.

[1] D. Mieloch (Ed.), Verification test report of MPEG immersive video, ISO/IEC JTC 1/SC 29/WG 04 N 0341, April 2023, Antalya.

At the 142^nd MPEG meeting, MPEG Video Coding (WG 04) promoted its MPEG immersive video (MIV) conformance and reference software standard (ISO/IEC 23090-23) to the Final draft international standard (FDIS) stage, the last formal milestone of its approval process. The document specifies how to conduct conformance tests and provides reference encoder and decoder software for ISO/IEC 23090-12 MPEG immersive video. This draft includes 23 verified and validated conformance bitstreams and encoding and decoding reference software based on version 15.1.1 of the Test model for MPEG immersive video (TMIV). The test model, objective metrics, and some other tools are publicly available at https://gitlab.com/mpeg-i-visual.

MIV was developed to support compression of immersive video content, in which a real or virtual 3D scene is captured by multiple real or virtual cameras. The standard enables the storage and distribution of immersive video content over existing and future networks, for playback with 6 degrees of freedom (6DoF) of view position and orientation. MIV is a flexible standard for multiview video with depth (MVD) and multi-planar video (MPI) that leverages the strong hardware support for commonly used video formats to code volumetric video. There are conformance bitstreams for all profiles, namely the MIV Main profile for MVD, the MIV Extended profile which enables MPI, and the MIV Geometry absent profile, which is suitable for use with cloud-based and decoder-side depth estimation. In addition to conformance testing, work on verification testing of MIV was finished. A formal subjective quality evaluation with “naïve” test subjects watching pre-defined pose traces in an immersive scene was performed. On average, MIV demonstrates a clear benefit over the previous state-of-the-art MPEG video standard for coding multiple views – the multiview extension of HEVC (MV-HEVC). According to the results, the average mean opinion score (MOS) savings of the MIV Main profile range from 1.20 to 4.69 (in 11-grade quality scale) for tested sequences, while for the MIV Geometry Absent profile the average savings was equal to 2.94.

Interdigital and Philips have collaborated to promote the MPEG-I standards ISO/IEC 23090-5 Video-based visual volumetric codec (V3C), ISO/IEC 23090-5 Annex H Video-based point cloud compression (V-PCC), ISO/IEC 23090-12 MPEG immersive video (MIV), and ISO/IEC 23090-14 Scene description at the AWE Europe conference [1] in Lisbon last week.

The promotion comprised:

• A booth on the exhibit with a set of related demos
• An oral presentation on the conference
• A press release about the collaboration
• Sponsorship of the conference

[1] https://www.awexr.com/

Leaflets: AWE_Leaflets.pdf

In July 2020, the Brazilian Terrestrial Television System Forum (SBTVD) issued a call for proposals to set the TV 3.0 standard [1], aiming at extending the current digital television system to new use cases and futuristic applications. Among these applications, it called for immersive (VR) codecs.

At the end of 2021, proposed MPEG V3C technologies (ISO/IEC 23090-5, Visual Volumetric Video-based Coding) were adopted to fulfill XR requirements for TV 3.0. The specification drafting is ongoing and complementary video real time tests (phase 3) are planned for H1/2023. The deployment of Brazil’s TV 3.0 system is planned for 2025.

Interdigital and Philips were present at the SET trade show and conference in Sao Paulo, Brazil (Aug 22nd to 25th 2022) to showcase technologies to support SBTVD standard 3.0 including immersive codecs. They share a booth with Fraunhofer (FhG).

The joint Interdigital/Philips demos were showcased, and panels were done for the promotion of the V3C standard towards many key players of Brazilian TV standards.

Flyer: Interdigital LinkedIn link, Philips link

The Motion Pictures Experts Group (MPEG), during its process of standardization, routinely conducts experiments to explore new technologies and to verify technologies that are introduced into a standard. The Video sub-group (WG04) also follows the same process. Experiments are performed on suitable video test material relevant to the experiment, where the test material are sourced from MPEG members and non-members alike. Without good test material, the outcomes of experiments could lead to inaccurate conclusions.

Currently, WG04, as part of its exploration activities, is performing experimentation on 6-DoF immersive video coding. In this activity, a three dimensional scene that is captured by numerous cameras (camera rig) is analysed and encoded. The arrangment of the cameras that capture the scene may have any arragement (regular/irregular). There is also no limitation placed on the inter-camera baseline distances; it may be narrow or wide. Each camera in the camera rig provides its captured videos in a known projection format, e.g. perspective or equirectangular.

While conducting exploration experiments on the above-listed topics, WG04 has realized that there is a dire need for more and appropriate test materials, especially omnidirectional (360 degree) 6DoF video content. There is also an immediate need for content that is captured/synthesized using non-planar rigs. Therefore, WG04 solicits new test materials from contributors. The call [1] describes the characteristics of test materials solicited, their formatting, and the process of contribution are described.

[1] Call for MPEG immersive video test materials, ISO/IEC JTC 1/SC 29/WG 04 N 0204, April 2022, Online.

The Motion Pictures Experts Group (MPEG), during its process of standardization, routinely conducts experiments to explore new technologies and to verify technologies that are introduced into a standard. The Video sub-group (WG04) also follows the same process. Experiments are performed on suitable video test material relevant to the experiment, where the test material are sourced from MPEG members and non-members alike. Without good test material, the outcomes of experiments could lead to inaccurate conclusions.

Currently, WG04, as part of its exploration activities, is performing experimentation on 6-DoF immersive video coding. In this activity, a three dimensional scene that is captured by numerous cameras (camera rig) is analysed and encoded. The arrangment of the cameras that capture the scene may have any arragement (regular/irregular). There is also no limitation placed on the inter-camera baseline distances; it may be narrow or wide. Each camera in the camera rig provides its captured videos in a known projection format, e.g. perspective or equirectangular.

While conducting exploration experiments on the above-listed topics, WG04 has realized that there is a dire need for more and appropriate test materials, especially omnidirectional (360 degree) 6DoF video content. There is also an immediate need for content that is captured/synthesized using non-planar rigs. Therefore, WG04 solicits new test materials from contributors The call [1] describes the characteristics of test materials solicited, their formatting, and the process of contribution are described.

[1] Call for MPEG Immersive video Test Materials, ISO/IEC JTC 1/SC 29/WG 04 N 0170, January 2022, Online.

At the 137th MPEG meeting, MPEG Video coding (WG 04) has promoted its MIV Conformance and Reference software standard (ISO/IEC 23090-23) to the Committee Draft (CD) stage. The document specifies how to conduct conformance tests, and it provides reference encoder and decoder software for ISO/IEC 23090-12 MPEG Immersive video. This draft includes 18 verified and validated conformance bitstreams, and reference software based on version 12.0 of the test model (TMIV). The test model, objective metrics and some other tools are publicly available at https://gitlab.com/mpeg-i-visual.

MIV was developed to support compression of immersive video content, in which a real or virtual 3D scene is captured by multiple real or virtual cameras. The standard enables storage and distribution of immersive video content over existing and future networks, for playback with 6 degrees of freedom of view position and orientation. MIV is a flexible standard for multiview video with depth (MVD) that leverages the strong hardware support for commonly used video codecs to code volumetric video. Views may use equirectangular, perspective or orthographic projection. By pruning and packing views, MIV is able to achieve bitrates around 15 to 30 Mb/s for HEVC and a pixel rate equivalent to HEVC Level 5.2. Besides the MIV Main profile for MVD, there is the MIV Geometry absent profile, suitable for cloud-based and decoder-side depth estimation, and the MIV Extended profile, which enables coding of multi-plane images (MPI). The MIV standard is designed as a set of extensions and profile restriction on the Visual Volumetric Video-based Coding and Video-based Point Cloud Coding (ISO/IEC 23090-5) standard. The conformance bitstreams span all profiles.

In addition to conformance testing, work on the verification test is ongoing, and the carriage of MIV is specified through the Carriage of V3C Data standard (ISO/IEC 23090-10). MPEG Requirements (WG 02) will publish final use cases and requirements for MIV edition-2, which will be an evolution of the MIV standard, and MPEG Liaison and Communication (AG 3) will publish the white paper on MIV.