Film Grain in Video Compression

Guides: Deepthi Nandakumar, Engineering Manager, x265, MulticoreWare Inc and Gayathri N, Asst. Professor, Amrita School of Engineering

INTRODUCTION

The visual quality of any natural video is lost when it is corrupted by grain. Film grain results from the physical granularity of the photographic emulsion. Film grain is most noticeable at high resolutions and on high-quality displays. Preservation of the grainy appearance of the film is desirable for artistic reasons. Film grain characteristics from different film stocks have a wide variety of size, shape, intensity, and color. In digital media, film grain management is a key tool. I try to implement a de-noising module which provides the best estimate of grain and improve visual quality. I also look into the aspect of designing custom quantization matrices which improves the performance of the encoder and improves visual quality.

Film grain has very high entropy and is not very compressible using block-based motion compensation prediction. Attempting to encode film grain often results in coding artifacts. The lack of film grain can create plastic looking images. Artifacts dependent on content and bit-rate: For low or medium bit-rates, it would be Flat images mostly missing film grain. For medium – high bit-rates, the video would be patchy, block grained. The grain is lost especially in areas of motion.The application is developed using C++, and MATLAB. The following modules are implemented for the application:

• Guided Filter
• Median Filter
• Canny Edge Detector
• Custom Quantization matrices generator

Quantization

Quantization, involved in image processing, is a lossy compression technique achieved by compressing a range of values to a single quantum value. When the number of discrete symbols in a given stream is reduced, the stream becomes more compressible. The human eye is fairly good at seeing small differences in brightness over a relatively large area, but not so good at distinguishing the exact strength of a high frequency (rapidly varying) brightness variation. This fact allows one to reduce the amount of information required by ignoring the high-frequency components. This is done by simply dividing each component in the frequency domain by a constant for that component, and then rounding to the nearest integer. This is the main lossy operation in the whole process. As a result of this, it is typically the case that many of the higher frequency components are rounded to zero, and many of the rest become small positive or negative numbers. As human vision is also more sensitive to luminance than chrominance, further compression can be obtained by working in a non-RGB color space which separates the two (e.g., YCbCr), and quantizing the channels separately.

HEVC allows customized scaling lists. The scaling lists are generated based on the DCT of the grain. As discussed earlier, the quantization matrix is designed to provide more resolution to more perceivable frequency components over less perceivable components. Hence, if the requirement is for de-noising, we decrease the resolution of the frequency components of the grain. The extent of decrease depends on the DCT coefficients of the grain as well as the position of the AC Transform Coefficients in the DCT matrix. Whereas, if the requirement is to preserve the grain, we enhance the resolution of the frequency components of the grain.

CONCLUSION

The de-noising module provides improved visual quality and better encoding. The new module enhances the performance, increases the bit rate without compromising on the visual quality.
The de-noising module uses highly efficient contemporary filters; hence, the computation time is minimum. The Custom quantization is also efficient as it does not add any other overhead to the existing system. The custom quantization matrices could be used for many additional features as well such as Feature enhancement, de-feathering etc. All the designs comply with the available legacy standards. The flexible design of the control makes the work of the developer easy in case of adding new sub-controls or modules in future without employing major code changes.

The following image shows the output of the algorithm when the degree of de-noising is set extremely high.