dm_dv(3dm)
dm_dv - DV and DVCPRO image and audio compression programming with dmIC, dmAC and dmBuffers
Showing IRIX 6.5.30 (default release). Last changed in IRIX 6.5.5.
NAME dm_dv - DV and DVCPRO image and audio compression programming with dmIC, dmAC and dmBuffers SYNOPSIS #include <dmedia/dm_imageconvert.h> #include <dmedia/audio.h> #include <dmedia/dm_audioutil.h> DESCRIPTION The DV and DVCPRO image and audio compression standards operate with dmIC and dmAC and dmBuffers as described in this man page. The dmIC and dmAC man pages are generic and contain no information about specific compression schemes. This man page describes details of dmIC specific to DV and DVCPRO image compression, and also describes DV and DVCPRO audio compression. This man page describes the difference between DV and DVCPRO compression, and makes recommendations as to when you should use each one. NOTE: some SGI DV and DVCPRO hardware is not supported by the dmIC interface yet. If you have a DIVO plugin board in an Octane or Onyx2, you will need to use VL library calls to access the hardware. However, the dmIC DV and DVCPRO software codecs will work fine on any machine, including ones that happen to have a DIVO board in them, but you will not get the additional hardware acceleration. WHICH TO USE: DV OR DVCPRO DV image compression and DVCPRO image compression are different, although they happen to share several important characteristics. They are both fixed bandwidth, at exactly 120,000 bytes per frame for NTSC and 144,000 bytes per frame for PAL. They both define exactly one frame size for NTSC (720x480 non-square pixels) and exactly one frame size for PAL (720x576 non-square pixels). Both DV and DVCPRO image frames leave space at the same place inside themselves for embedded audio (described below). However, DV is different from DVCPRO in several very important respects. Simply put, they are entirely different compression schemes. For example, if you attempt to use a DV decoder to decode a frame created by a PAL DVCPRO camcorder or deck, it simply will not work because they are completely different. And vice versa: if you attempt to decode a PAL DV frame with a DVCPRO deck it will not work. Luckily, DV and DVCPRO have built in headers that identify each frame as either DV or DVCPRO. If you have a file containing some sort of DV or DVCPRO data and you want to know which it is, the "dminfo" command will tell you exactly which you have. DV and DVCPRO also specify different audio compression. Or more precisely, DVCPRO audio is a perfect subset of DV audio. DVCPRO is very restrictive, only allowing 2-Channel 48 KHz "locked mode" 16 bit linear audio. DV allows three audio rates: 32 KHz, 44.1 KHz, and 48 KHz, and not only has mono and 2-Channel audio, but also allows 16 different varieties of 4-Channel 12 bit non-linear audio. In conclusion, it is very simple when to create DV data and when to create DVCPRO data. If your eventual delivery target is a DV camcorder or deck (the equipment might say it is a "DV" or "miniDV" or "DVCAM"), create DV data. If your eventual delivery target is a DVCPRO camcorder or deck (the equipment might say DVCPRO, or SMPTE D-7), create DVCPRO data. If you are in doubt, you should probably create DV data which is the more common format. THE DIF FILE FORMAT Although you can embed DV and DVCPRO compressed data in several wrapper formats such as QuickTime or AVI, there is also a standard file format for DV and DVCPRO compressed data called DIF. DIF is very simple: it is one or more compressed frames appended end-to-end. If the data is NTSC size, it must be played back at exactly 29.97 frames per second, and if the data is PAL size, it must be played back at exactly 25.0 frames per second. There is a convenient SGI MovieLib function to convert any arbitrarily complex movie to a DIF stream called mvExportFlattenedFile(3dm). Do a man on that function to see how to create a DIF file in only a very few lines of code. IMAGE PACKING The image packing inside the compressed DV and DVCPRO frames are as follows (packing enums are found in the header file dmedia/dm_image.h): NTSC-size DVCPRO is 4:1:1 packed (enum DM_IMAGE_PACKING_YCrCbYYY) NTSC-size DV is 4:1:1 packed (enum DM_IMAGE_PACKING_YCrCbYYY) PAL-size DVCPRO is 4:1:1 packed (enum DM_IMAGE_PACKING_YCrCbYYY) PAL-size DV is 4:2:0 packed (enum DM_IMAGE_PACKING_YCrYYCrYYCbYYCbY) In a program that is decompressing DV or DVCPRO with dmIC, it is recommended that the destination be specified as uncompressed with 4:2:2 packing (enum DM_IMAGE_PACKING_CbYCrY). This is the fastest possible decode path in both the dmIC software decoding module and in the hardware accelerated dmIC decoding module. In addition, 4:2:2 packing can be sent to the workstation's video out jacks efficiently, and also painted to graphics efficiently using glDrawPixels(3G). If the data is needed in non-YUV form, the destination can be requested as XRGB (enum DM_IMAGE_PACKING_XRGB) or other such RGB packing formats. It is illegal to ask for the data to be decompressed to either 4:1:1 or 4:2:0 packings, as these are not supported uncompressed packings. AVAILABLE DV AND DVCPRO CONVERTERS As of IRIX 6.5, DV and DVCPRO image converters are available on all SGI platforms through the dmIC API, and realtime hardware support is available only on the O2 through the dmIC API. Also as of IRIX 6.5, DV and DVCPRO audio converters are available on all SGI platforms through the dmAC API. DV and DVCPRO audio can be encoded and decoded in realtime in software, and no hardware support is available. FINDING AND CREATING IMAGE CONVERTERS This section describes how to use dmIC operations to find and create a DV or DVCPRO image converter for either encode (compression) or decode (decompression) in realtime or non-realtime. (Realtime generally refers to a DV or DVCPRO converter that operates at video rate -- NTSC, PAL, or 601). A DV or DVCPRO image converter's DM_IC_ID parameter -- from the DMparams list returned by dmICGetDescription -- has the value 'dvc '. (In other words, it is an unsigned int where the most significant byte is the ascii representation of the character 'd', the next byte is 'v', and so on.) The DM_IC_SPEED parameter will have the value DM_IC_SPEED_REALTIME if the converter is capable of video rate processing. If the value is DM_IC_SPEED_NONREALTIME the converter is not capable of operating at video rate. The DM_IC_CODE_DIRECTION parameter indicates if the converter compresses or decompresses. If the value is DM_IC_CODE_DIRECTION_DECODE then the input to the converter is DV or DVCPRO compressed data, and the output is pixel data (uncompressed). If the value is DM_IC_CODE_DIRECTION_ENCODE then the input to the converter is pixel data and the output is DV or DVCPRO depending on what you request. The following code fragment shows how to find a realtime DV or DVCPRO image decoder and create a context for operating on it: DMimageconverter ic; DMparams *p; int n; for (n=0; n < dmICGetNum(); n++) { dmParamsCreate(&p); if (dmICGetDescription(n, p) != DM_SUCCESS) continue; if ((dmParamsGetInt(p, DM_IC_ID) == 'dvc ') && (dmParamsGetEnum(p, DM_IC_SPEED) == DM_IC_SPEED_REALTIME) && (dmParamsGetEnum(p, DM_IC_CODE_DIRECTION) == DM_IC_CODE_DIRECTION_DECODE)) { dmParamsDestroy(p); break; } dmParamsDestroy(p); } dmICCreate(n, &ic); The number of simultanously active realtime DV and DVCPRO converters system-wide may exceed the capabilities of the underlying hardware. That is, one or more programs may create one or more DV or DVCPRO converter contexts and if too many are simultaneously compressing or decompressing DV data the result may be less than realtime for one or more contexts. DV AND DVCPRO CONVERTER CONTROL These sections describe how generic and DV-specific parameters are used to control a DV or DVCPRO converter. In general each parameter value should be considered undefined until set explicitly by the program. The input image format, the output image format and the conversion operation are controlled using dmICSetSrcParams, dmICSetDstParams, and, dmICSetConvParams repectively. The following discussion will use the terms source and destination to refer to the image format of the converter input and output, respectively. The terms compressed side and uncompressed side will generally refer to parameters of the source and destination respectively if the converter is a DV or DVCPRO decoder and vice versa if the converter is a DV or DVCPRO encoder. DV AND DVCPRO SRC/DST IMAGE FORMAT The image orientation, width, height, and pixel format must be set on both the source and destination side of the converter. Use of dmSetImageDefaults is recommended to set the DM_IMAGE_WIDTH, DM_IMAGE_HEIGHT, and DM_IMAGE_PACKING parameters. The width and height on the compressed side must match the native size of the DV or DVCPRO encoded data, which is exactly 720x480 for NTSC size data or 720x576 for PAL size data. The DM_IMAGE_COMPRESSION parameter must be set to either DM_IMAGE_DV or DM_IMAGE_DVCPRO on the source or destination side as appropriate for the conversion direction. DV and DVCPRO image converters are either DM_IMAGE_PACKING_YCrCbYYY or DM_IMAGE_PACKING_YCrYYCrYYCbY packed on the compressed side (see the section in this man page on IMAGE PACKING). For the highest performance, always choose DM_IMAGE_PACKING_CbYCrY (4:2:2) on the destination side, which is the same as the Video Library pixel packing for 4:2:2 video VL_PACKING_YVYU_422_8, and is the same as the OpenGL GL_YCRCB_422_SGIX format for use with glDrawPixels on some machines (O2, for example). The following code fragment configures the source and destination of the realtime decoder context created in the previous example, and assuming DM_IMAGE_DV NTSC-size data. Error processing is left off for clarity: DMparams *p; size_t uncompressedImageSize; dmParamsCreate(&p); dmSetImageDefaults(p, 720, 480, DM_IMAGE_PACKING_YCrCbYYY); dmParamsSetEnum(p, DM_IMAGE_ORIENTATION, DM_IMAGE_TOP_TO_BOTTOM); dmParamsSetString(p, DM_IMAGE_COMPRESSION, DM_IMAGE_DV); dmICSetSrcParams(ic, p); dmParamsSetString(p, DM_IMAGE_COMPRESSION, DM_IMAGE_UNCOMPRESSED); dmSetImageEnum(p, DM_IMAGE_PACKING, DM_IMAGE_PACKING_CbYCrY); uncompressedImageSize = dmImageFrameSize(p); dmICSetDstParams(ic, p); dmParamsDestroy(p); In the above code note that the DMparams list p is recycled since only two parameter values are different between the source and destination formats. This is safe since dmIC saves away the values of the paramters of interest before returning from any call accepting a DMparams as an argument. Also note the use of dmImageFrameSize to compute the size (in bytes) of an uncompressed image described by the given parameters. dmImageFrameSize DOES NOT work correctly for any compressed image sizes. This is because compressed sizes of images are data dependant for most compression codecs. Luckily, in compressed DV or DVCPRO NTSC-size frames are always exactly 120000 bytes, and PAL-size compressed frames are always exactly 144000 bytes. The orientation in this example is DM_IMAGE_TOP_TO_BOTTOM which is the proper orientation of video imagery and is the value required for DM_IMAGE_ORIENTATION to ensure realtime processing. WARNING: A realtime DV or DVCPRO converter may not necessarily operate in realtime if one or more of the image parameters (other than DM_IMAGE_COMPRESSION and DM_IMAGE_PACKING) is different between source and destination. That is, the implied conversion (from one width and height to a different if those paramters are different) may take place in software. DV AND DVCPRO CONVERSION CONTROLS Aside from the rudimentary image conversion controls on width, height, pixel packing, orientation and so forth there are controls specific to the DV encode process typically to control the image quality. This section describes these parameters for use with dmICSetConvParams. There are two supported qualities for compression, and you choose between them by using the DM_IMAGE_QUALITY_SPATIAL parameter. If this parameter value is set to less than 0.5, the lower quality compression which is faster is used, and if this parameter value is higher than 0.5 the higher quality compression is used. Higher quality slows down the time it takes to compress each frame, but results in better picture quality. DV AUDIO USING DMAC DV and DVCPRO Audio can be encoded and decoded using the dmAC audio conversion libraries. For encoding DV and DVCPRO audio, the most common scenario is that your program would first compress a frame of image, and then hand that compressed frame of image to the dmAC audio compression library along with a buffer of uncompressed audio. The dmAC audio compression library carefully compressed the audio and "knits" it into the correct location inside the already compressed image. For decoding DV and DVCPRO audio, the most common scenario is that your program would read a frame of knitted audio and video from disk into a buffer, and hand off that buffer first to the dmIC functions to uncompress the image portion, then hand the same buffer off to the dmAC functions to uncompress the audio. Below is an example piece of code which assumes a DIF file as input, determines whether it is NTSC or PAL sized data, and finally sets up and calls the dmAC functions to extract and uncompress the audio from the first frame. Notice that this handles all possible cases of DV and DVCPRO compressed audio, both NTSC and PAL size, automatically converting it to 16 bit, uncompressed, 2 channel, 48 kHz audio. (Error checking is left off for clarity.) #define NTSC_FRAME_SIZE 120000 #define PAL_FRAME_SIZE 144000 FILE *fp; DMparams *srcParams, *dstParams; DMaudioconverter audioConverter; void *difFrame = NULL, *outputAudioBuf = NULL; int numSrcFrames, numBytesOfInput, numSamplesOfOutput; int compressedDataSize; /* * read a large enough chunk off the front of the file to get either * a complete NTSC size frame or PAL size frame (PAL is larger) */ fp = fopen(argv[1], "r"); difFrame = (void *) malloc(PAL_FRAME_SIZE); fread(difFrame, PAL_FRAME_SIZE, 1, fp); /* * set up the src parameters to exactly reflect what is is inside * this particular DIF frame. */ dmParamsCreate(&srcParams); dmDVAudioHeaderGetParams(difFrame, srcParams, &numSrcFrames); dmParamsSetString(srcParams, DM_AUDIO_COMPRESSION, DM_AUDIO_DV); if (dmParamsGetInt(srcParams, DM_DVAUDIO_FORMAT) == DM_DVAUDIO_NTSC) compressedDataSize = NTSC_FRAME_SIZE; else /* DM_DVAUDIO_FORMAT == DM_DVAUDIO_PAL */ compressedDataSize = PAL_FRAME_SIZE; /* * set up the dst parameters to produce a buffer of 48 kHz, 16 bit * samples, and 2 channels. The dmAC library will do all * conversion for us automatically. Also, allocate a buffer large * enough to contain all the samples removed from the first frame * and converted to this format. (2 channels, 16 bit = 2 byte) */ dmParamsCreate(&dstParams); dmSetAudioDefaults(dstParams, 16, 48000, 2); outputAudioBuf = (void *) malloc(2 * 2 * numSrcFrames); /* * create the converter, and configure it. */ dmACCreate(&audioConverter); dmACSetParams(audioConverter, srcParams, dstParams, NULL); /* * Now call the dmAC converter, asking it to extract all the audio * embedded in the first frame and convert it to our requested * format. */ numBytesOfInput = compressedDataSize; numSamplesOfOutput = numSrcFrames; dmACConvert(audioConverter, difFrame, outputAudioBuf, &numBytesOfInput, &numSamplesOfOutput); printf("dmACConvert took %d bytes from input\n", numBytesOfInput); printf("dmACConvert wrote %d samples to outbuf\n", numSamplesOfOutput); The outputAudioBuf produced by this call to dmACConvert could then be sent to the audio hardware to eventually come out the speakers, or used for some other purpose. The format of the output buffer is LRLRLR... where each "L" is 16 bits and each "R" is 16 bits. The output buffer must be played at 48 kHz to sound correct. ENCODING DV AND DVCPRO AUDIO Encoding DV or DVCPRO audio is very similar to the example code given in the section above, but with the srcParams and dstParams switched. In addition, it is very important to set the dmParam "DM_DVAUDIO_TYPE" to be the correct one of either DM_DVAUDIO_DV or DM_DVAUDIO_DVCPRO. The reason is that the audio encoder must set the internal header state correctly, and that header is shared between the image and audio compression. So if you have encoded DVCPRO image data, you will erase the correct header and replace it with an incorrect header if you do not specify the DM_DVAUDIO_TYPE as DM_DVAUDIO_DVCPRO. SEE ALSO dmIC(4), dmACConvert(3dm), dmBuffer(4), dmParams(3dm), dmICCreate(3dm), dmICChooseConverter(3dm), dmICGetDescription(3dm), mvIntro(3dm), mvExportFlattenedFile(3dm), dmDVAudioHeaderGetParams(3dm), dmDVAudioDecode(3dm), dmDVAudioDecoderCreate(3dm), dmDVAudioDecoderDestroy(3dm), dmDVAudioDecoderGetParams(3dm), dmDVAudioDecoderReset(3dm), dmDVAudioDecoderSetParams(3dm), dmDVAudioEncode(3dm), dmDVAudioEncoderCreate(3dm), dmDVAudioEncoderDestroy(3dm), dmDVAudioEncoderGetParams(3dm), dmDVAudioEncoderGetFrameSize(3dm), dmDVAudioEncoderReset(3dm), dmDVAudioEncoderSetParams(3dm).