Media Analysis

Similar technology concerns affect the import and analysis of media files.

QuickTime importer

For a long time the default, if not only, mechanism to analyse arbitrary media files (to extract technical metadata like frame size, duration, video and audio codec, timecode, as well as any embedded metadata like title, comments, copyright, markers) was QuickTime. Initially this was via QuickTime for Java, then in CatDV 11 support was added to the Native Helper so we would read the same metadata using the native QuickTime APIs.

The advantage of QuickTime is that it supports a very wide range of media files, not just movies but also audio only files, still images, etc., and that it allows 3rd party extensions and codecs, such as those from Calibrated or Avid or Flip4Mac, to be installed to support formats like MXF and WMV that QT doesn’t support natively.

The disadvantage of QuickTime is that it is 32-bit only and hasn’t been updated by Apple for many years, so MXF in particular is only supported through 3rd party MXF

import components. It has also been deprecated on Windows, and in time will no doubt disappear on the Mac also.

(This is the original QuickTime 7 API we are talking about, by the way. QuickTime Player X is slightly newer, and sits on top of both QuickTime 7 and AVFoundation and uses various bridging technologies, so it can play more files, but it doesn’t provide a useful API for applications like CatDV to use.)

Pure Java importers

While quite a lot of embedded metadata is translated and made available via the QuickTime API, not all is. For this reason, and to reduce our dependency on QuickTime, we also have pure-Java parsers that will directly read and extract basic metadata from most common media file formats:

· JQT parser reads MOV, MP4 and J2K (JPEG2000) files

· AVI parser reads AVI, WAV and BWF files

· MPEG parser reads MPG, MP2, MPA, 3GP, MTS, TS and M2TS files (and is also used to read additional format information from H264 streams in a MOV container)

· MXF parser

· ASF parser for WMV and WMA files

· JavaImage parser reads JPG, TIF, BMP, PSD, PNG, GIF and other still image files

· TIFF parser reads TIF and many digital still camera RAW formats (CR2, NEF, DNG, SRF, PEF, ARW, RAF, etc.)

· DV parser reads metadata from DV, DVCAM, DVCPro files (including when they are wrapped in a MOV or MXF container)

· DPX parser for DPX images

· EXR parser for EXR images

· PDF parser for PDF documents

· JPEG parser for JPG images, and also to extract JPEG encoded thumbnails that are often embedded in other files such as RAW camera formats, PDF documents, DOCX or PPTX files, etc.

· RED parser for R3D files (we also use the official RED SDK via the native helper to read some metadata that we’re unable to decode ourselves)

· OMF parser (OMF files are a legacy format that Avid once used and a forerunner to MXF files)

· OOXML parser for reading DOCX, PPTX and XSLX Office documents

These low level file parsers work in conjunction with, or as an alternative to, using QuickTime to read the files and are controlled via various import preferences.

On top of these “parsers” we also have “importers” that detect complex camera card folder structures like XDCAM, P2, XAVC etc. and create the appropriate metaclips.

FFmpeg importer

In CatDV 12.1 we added a new importer that uses FFmpeg’s ability to report metadata such as the media file duration, separate streams within it, technical format (codec, sample rate, bit rate, etc.) of each stream, timecode, etc. This is required for importing media files for which no pure Java parsers exist when QuickTime isn’t available, most notably audio formats such as MP3 and AIFF.