Introduction
Ever since file based broadcast video production became a practical reality in the mid-late 1990s there has been a drive to define some file standards which can enable true seamless interoperability.
The first video servers and editing systems used proprietary standards, such as GXF from Grass Valley, LXF from Leitch, AAF from Avid, QuickTime from Apple. In the later part of the 1990s, a consortium of broadcasters and manufacturers collaborated to produce the MXF (Material Exchange Format) set of file format standards.
This was a big step towards the goal of true interoperability. However, MXF- by necessity- allows a great deal of flexibility and therefore room for incompatibility. For example, it defines several container formats in whic
h the video and audio essence can be whatever any company decides is best for their use. In 2007 the Advanced Media Work ow Association (AMWA) was formed out of what was previously known as the AAF Association. One of the goals of AMWA was to better agree and define working standards that constrain MXF to satisfy specific operational requirements. These are now published as “Application Specifications”.
Currently there are 6 published AMWA AS models to suit different production needs:
AS-02: Program Mastering when there are multiple versions of the content
AS-03: Program Delivery to multiple broadcast stations
AS-07: Archive and Preservation of a wide range of material
AS-10: Workflows suited to fast turnaround end-to-end production
AS-11: Media Contribution for the delivery of finished media assets to a broadcaster or publisher
AS-12: The use of metadata for the management of content, including a unique identifier
Even these models are not constrained tightly enough for some use cases, so in the last few years we have seen the emergence of several delivery specifications designed to suit the needs of different national broadcaster groups.
The first of these, the Digital Production Partnership (DPP) was formed in the UK in 2013. The DPP defined a specification for content being delivered to UK broadcasters, based on AMWA AS-11, but with a “shim” which further constrains the video and audio encoding parameters. DPP also defines the layout (known as lineup in the UK) of the content to be delivered. It specifies the precise time line and properties of color bars, clock (slate), black frames and tone that must precede each programme or part of a programme. It even defines the motion of the second hand on the clock. DPP goes further and also defines the metadata that must be delivered with the material.
We have seen similar delivery standards come from the French broadcast community, based on AMWA AS-10, but also with some additional specifications. There is also a growing number of companies (most notably Disney and Net ix) wishing to use the Interoperable Mastering Format (IMF) which is also based on MXF.
In 2015 the two national broadcasters in Germany – ARD and ZDF – cooperated to define a set of standards called ARD_ZDF_MXF. These standards are based on MXF, with explicit specifications which suit the requirements of the German public broadcasters for file interchange between production centres. This new format, often referred to as HDF, will be the required format for interchange of all media among ARD and ZDF companies and sites from 1 January 2017, and a number of broadcasters in other countries are also planning to use it.
All of these delivery standards are designed to improve production efficiency for the broadcasters. This article will explain how broadcasters and production companies can join this drive for efficiency with automated tools for creation of ARD_ZDF_HDF files, and for proving that their content complies with this precise and exacting standard, and includes examples of broadcasters in Germany who are already using these tools.
ARD_ZDF_HDF Technical Overview
Unlike DPP, the ARD_ZDF_HDF speci cation (HDF refers to High Definition Format) does not mandate the layout of the video or the metadata. It only defines the way that the MXF file is encoded.
Within the HDF specification there are 6 profiles allowed to suit different production requirements.
Profiles HDF 01a and 01b support XDCAMHD, which is the most commonly exchanged and production format used in Germany. HDF-02 and HDF-03 define profles for AVC-Intra which are also used by many broadcasters, primarily in transmission though also with some production uses.
Within each of these profiles there are a number of detailed and specific requirements for the way in which the MXF file is constructed, beyond the obvious parameters. For example the HDF 01a and 01b profiles require that the video and audio tracks in the file are segmented into Body Partitions of 10 seconds duration and that a full index table be included in the Footer Partition of the file
Through the IRT, reference files are available to show companies exactly how each of these profiles should be encoded, and the IRT has established a testing process which equipment manufacturers can use to obtain certification that the products , typically encoders and transcoders, conform to the specification.
With effect from 1 January 2017, any German broadcaster or production company should use an IRT certified product for encoding of compliant HDF files without risk or compromise.
Automated Encoding and Production Examples
Telestream Vantage is the industry’s leading transcoding and workflow automation solution, and is already in widespread use for many applications throughout German broadcaster groups. Prominent users of Vantage include NDR, NDR Regions, MDR, BR, WDR, SWR, RBB, HR, ZDF, all of whom have been actively providing feedback to Telestream on their requirements for HDF encoding.
For any encoder or transcoder, creating an MXF file is a basic function that a competent user can configure correctly, given enough time and patience. However, configuring an encoder to exactly meet the specifications for the HDF profiles is not a trivial task. Within Vantage Transcode Pro, Telestream has taken the guesswork and risk out of this problem by creating predefined templates for all of the HDF profiles, and has submitted sample files of these to the IRT for testing and expects to have these certificates of compliance before the end of 2016.
Now, to create any specific HDF profile and deliver that to a broadcaster, can be as simple as dropping a file into a watch folder anywhere in the network, then Vantage work ow management system does the rest.
The same process could be triggered through API calls by any MAM system, or even directly from any edit system user interface. In any of these cases, the risk of operator error is removed from the process.
Automated QC and Compliance Checking
Although most production departments will use a certified encoder and workflow management system to create compliant HDF files, there will be situations where external production companies deliver files, or where content is received from outside of Germany, or even when content is restored from archive systems- and in these cases there could be some doubts as to the compliance of the file with HDF standards.
For all of these cases, an automated QC test system is an essential tool for broadcasters to check incoming files and for production companies to check their deliveries to broadcasters. The prime objective of such as system is to quickly and reliably check that a file complies with the required HDF profile, and in case of failures, to highlight why it may not comply.
Unlike the case for encoders, the IRT does not (at present) offer a certification service for automated QC systems. Fortunately, they do provide the aforesaid reference files and this allows the Telestream Vidcheck- er QC system to create a test plan based on the reference files using its unique “Auto-Template” feature.
Templates are already provided in Vidchecker for the most commonly used profiles (HDF 01a and HDF 01b), while others can be created in minutes based on the IRT reference files.
When choosing an automated QC system, users should look for solutions that produce pdf certificates that can be delivered with their files as proof of compliance. Equally, they should consider the level and quality of information p
rovided when files do not fully comply. Vidchecker provides fail and warning alerts which can be customized to suit the users language and preferred terminology, along with operator tools that allow expert users to view the file at the exact point of an alert so that they can decide if this was a real error or is not significant enough to reject the file.
Summary
The implementation of HDF standards in ARD, ZDF and other broadcasters will lead to much greater interoperability and will simplify the process of supplying, distributing and exchanging media files in Germany and other countries. To take full advantage of this standardization, both broadcasters and production companies should check that their encoders are certified by the IRT as HDF compliant, and should also ensure that they have an automated QC system which can not only test for compliance but which can easily help them to under- stand the causes of any non-compliance alerts
Click here to read this in German.
Pingback: Efficient Production and Testing for Compliance with ARD_ZDF_HDF File Standards
Hey great post. I hope it’s ok that I shared this
on my Twitter, if not, no worries just let me know
and I’ll remove it. Either way keep up the good work.