5 Video Communication Data Type

5.1 Introduction

Video signals are communicated between processes by means of tokens that travel over fifo channels. Modules are seamlessly plugable when there is a match between the number of I/O ports, the signal types (e.g. Y,U,V) communicated through these ports, and the data types of the tokens that make up each signal. The choice of the latter, which is referred to as the communication data type, determines the execution speed, the memory usage, and the re-usability of YAPI code [1],[3]. Our primary goal is maximum re-usability, which requires that the application defines exactly what is communicated, but not how: it is the mapping of a specification to an implementation that decides how the communication is done most efficiently. This goal is supported by communicating the smallest possible token size, which, for video processing, is a pixel. Communicating pixels allows the application programmer to choose his processing to be on a pixel, line, field or frame basis, simply by reading and writing vectors of pixels. YAPI supports efficient vector read/write methods for this purpose [1].

5.2 Definitions

All VYA definitions reside in a file called vya.h; they are step by step introduced below. To begin with, video pixels are of type VYAsigned16bit:

typedef signed16bit VYApixel;

The implementation of type VYAsigned16bit is a design decision which is independent of VYA and must be specified in a separate file called architecture.h. For example:

typedef short VYAsigned16bit;

In computations on type VYApixel, VYA assumes signed data ranging from -215 to 215-1. In all implementations, type VYAsigned16bit must therefore contain at least 16 bits. See also section 8. Vectors of pixels either represent one-dimensional arrays called lines or two-dimensional, rectangular arrays called images. The number of pixels in a line is indicated by VYAlineLength. As it may vary for each line that is communicated, VYAlineLength must be supplied for each line.

typedef unsigned int VYAlineLength;

The size of an image is defined by VYAimageWidth which indicates the (fixed) number of pixels per line, and VYAimageHeight which is the number of lines per image. Both must be supplied once for each image. Finally, an image can be either a top field, a bottom field, or a frame. This is indicated by VYAimageType, to be supplied per image when needed by the application.

typedef unsigned int VYAimageWidth;
typedef unsigned int VYAimageHeight;
enum VYAimageType {VYA_TOP_FIELD,VYA_BOTTOM_FIELD,VYA_FRAME};

5.3 Interfaces

The re-usability of any YAPI process or process network improves when it communicates only information that is required for proper operation. A gamma correction process for instance operates on individual pixels and does not need the notion of lines and images; it should therefore not receive any VYAlineLength, VYAimageWidth, VYAimageHeight or VYAimageType tokens. Similarly, a horizontal filter does not need the notion of images, etc. VYA therefore distinguishes between pixel-based, line-based and image-based input and/or output interfaces, as shown in Figure 1 - Figure 3.

Figure 1: VYA interfaces for modules with pixel-based input and/or output

Figure 2: VYA interfaces for modules with line-based input and/or output

Figure 3: VYA interfaces for modules with image-based input and/or output

Notes:

• "C" stands for "Component" and indicates that the interface can be used for any signal type.
• For line-based modules with both U and V inputs or outputs, it is assumed that these signals have the same line length.
• For image-based modules with both U and V inputs or outputs, it is assumed that these signals have the same image size and image type.
• For image-based modules with Y, U and V inputs or outputs, it is assumed that all three signals are of the same image type; Y and U,V may be of different size, however.
• For image-based modules with R, G and B inputs or outputs, it is assumed that all three signals are of the same size and image type.
• Output interfaces for line length, image size and image type should only be provided for those parameters that can be modified by the module.
• Modules may have additional interfaces, e.g., for control purposes.

5.4 Conversions

A system composed of VYA modules needs converters when it contains both modules with line-based and modules with image-based interfaces. These converters are shown in Figure`4 below. Module "i2l" converts VYAimageWidth to VYAlineLength by creating VYAimageHeight output tokens for each input token it receives. Likewise, module "l2i" converts VYAlineLength to VYAimageWidth by reading VYAimageHeight input tokens and producing a single output token.

Figure 4: VYA modules for converting between line-based and image-based interfaces.

Figure 5 shows an example application of "i2l". Note that it reveals a need for multicasting, i.e., connecting multiple fifos to a single output port. This will be supported by later YAPI versions.

Figure 5: Example application of image-to-line conversion.