Call for MPEG immersive video test materials

The Motion Pictures Experts Group (MPEG), during its process of standardization, routinely conducts experiments to explore new technologies and to verify technologies that are introduced into a standard. The Video sub-group (WG04) also follows the same process. Experiments are performed on suitable video test material relevant to the experiment, where the test material are sourced from MPEG members and non-members alike. Without good test material, the outcomes of experiments could lead to inaccurate conclusions.

Currently, WG04, as part of its exploration activities, is performing experimentation on 6-DoF immersive video coding. In this activity, a three dimensional scene that is captured by numerous cameras (camera rig) is analysed and encoded. The arrangment of the cameras that capture the scene may have any arragement (regular/irregular). There is also no limitation placed on the inter-camera baseline distances; it may be narrow or wide. Each camera in the camera rig provides its captured videos in a known projection format, e.g. perspective or equirectangular.

While conducting exploration experiments on the above-listed topics, WG04 has realized that there is a dire need for more and appropriate test materials, especially omnidirectional (360 degree) 6DoF video content. There is also an immediate need for content that is captured/synthesized using non-planar rigs. Therefore, WG04 solicits new test materials from contributors. The call [1] describes the characteristics of test materials solicited, their formatting, and the process of contribution are described.

[1] Call for MPEG immersive video test materialsISO/IEC JTC 1/SC 29/WG 04 N 0204, April 2022, Online.

Call for MIV test materials

The Motion Pictures Experts Group (MPEG), during its process of standardization, routinely conducts experiments to explore new technologies and to verify technologies that are introduced into a standard. The Video sub-group (WG04) also follows the same process. Experiments are performed on suitable video test material relevant to the experiment, where the test material are sourced from MPEG members and non-members alike. Without good test material, the outcomes of experiments could lead to inaccurate conclusions.

Currently, WG04, as part of its exploration activities, is performing experimentation on 6-DoF immersive video coding. In this activity, a three dimensional scene that is captured by numerous cameras (camera rig) is analysed and encoded. The arrangment of the cameras that capture the scene may have any arragement (regular/irregular). There is also no limitation placed on the inter-camera baseline distances; it may be narrow or wide. Each camera in the camera rig provides its captured videos in a known projection format, e.g. perspective or equirectangular.

While conducting exploration experiments on the above-listed topics, WG04 has realized that there is a dire need for more and appropriate test materials, especially omnidirectional (360 degree) 6DoF video content. There is also an immediate need for content that is captured/synthesized using non-planar rigs. Therefore, WG04 solicits new test materials from contributors The call [1] describes the characteristics of test materials solicited, their formatting, and the process of contribution are described.

[1] Call for MPEG Immersive video Test MaterialsISO/IEC JTC 1/SC 29/WG 04 N 0170, January 2022, Online.

Main outcomes of the 137th MPEG meeting (January 2022)

At the 137th MPEG meeting, MPEG Video coding (WG 04) has promoted its MIV Conformance and Reference software standard (ISO/IEC 23090-23) to the Committee Draft (CD) stage. The document specifies how to conduct conformance tests, and it provides reference encoder and decoder software for ISO/IEC 23090-12 MPEG Immersive video. This draft includes 18 verified and validated conformance bitstreams, and reference software based on version 12.0 of the test model (TMIV). The test model, objective metrics and some other tools are publicly available at https://gitlab.com/mpeg-i-visual.

MIV was developed to support compression of immersive video content, in which a real or virtual 3D scene is captured by multiple real or virtual cameras. The standard enables storage and distribution of immersive video content over existing and future networks, for playback with 6 degrees of freedom of view position and orientation. MIV is a flexible standard for multiview video with depth (MVD) that leverages the strong hardware support for commonly used video codecs to code volumetric video. Views may use equirectangular, perspective or orthographic projection. By pruning and packing views, MIV is able to achieve bitrates around 15 to 30 Mb/s for HEVC and a pixel rate equivalent to HEVC Level 5.2. Besides the MIV Main profile for MVD, there is the MIV Geometry absent profile, suitable for cloud-based and decoder-side depth estimation, and the MIV Extended profile, which enables coding of multi-plane images (MPI). The MIV standard is designed as a set of extensions and profile restriction on the Visual Volumetric Video-based Coding and Video-based Point Cloud Coding (ISO/IEC 23090-5) standard. The conformance bitstreams span all profiles.

In addition to conformance testing, work on the verification test is ongoing, and the carriage of MIV is specified through the Carriage of V3C Data standard (ISO/IEC 23090-10). MPEG Requirements (WG 02) will publish final use cases and requirements for MIV edition-2, which will be an evolution of the MIV standard, and MPEG Liaison and Communication (AG 3) will publish the white paper on MIV.