CHAPTER 11 : USING COLOUR

The previous chapters have presented several processes for reconstructing degraded monochrome data. No real consideration has been given to the use of colour information. This is natural since much of the archived film and video material is monochrome. However, colour film and television have been in use for at least twenty years and so there is also a substantial amount of archived colour material. Furthermore modern film and video media are also subject to degradation, be it digital dropout or poor recording and transmission conditions. It is traditional to consider that the processing of colour images can proceed by manipulating each colour component separately. This mode of colour algorithm implementation is quite successful and most commercially available video manipulation hardware and software currently process colour data in this way.

Despite this success, many authors have recognized that it would be best to use colour information in a multichannel framework for signal estimation. In this mode the data is treated as vector valued samples of a spatio-temporal signal. This chapter outlines the adjustments necessary to apply some of the algorithms discussed in this book to multichannel (colour) data. The work is still in progress and the pictures at the end of the chapter mainly illustrate system performance using separate processing of each channel.

Missing data removal from VOITURE

See Figure 11.1 in the book. The images below are better able to illustrate the differences between the sampled result and median interpolation. There is one more picture here than is shown in the book. It is shown in the third row on the left as the Least Squares interpolant. It may still be difficult to see a difference with the image on the right, which is the sampled interpolant, but the reader can view all these images using the .PPM files in this directory. The sampled interpolant generally blends more seamlessly with the background image texture. The Least Squares or MAP interpolant tends to stand out as a `flatter' area.

Removing Missing Data from VOITURE.
Dirty BADVOIT MEDVOITN
MEDVOITS SMPVOITS TIIR2
MAP reconstruction MAPVOITS Sampled reconstruction SMPVOITS
Zoom on Median result SUBMED Zoom on sampled result SUBSMP



The images below show more clearly the points illustrated by Figure 11.2 in the book.

Reconstructing large missing areas in PULL

Removing Missing Data from PULL.
PULL1
Three frames from dirty PULL.SEQ. Top to bottom: PULL1, PULL2, PULL3. Restored using VML3Dex as described in the book. Top to Bottom: RESV1, RESV2, RESV3. Restored using iterative algorithm as described in the book. Top to Bottom: RESO1, RESO2, RESO3.

Sequences on the CD-ROM

There is only one clip shown in colour on the CD-ROM, it is the sequence showing extremely severe corruption: PULL.SEQ. The data is stored in raw binary format, each frame is 720 pixels wide by 576 pixels high. Each colour image is represented by three frames of red, green and blue images which follow each other consecutively in the file. Thus for a sequence with 10 images there are 30 frames of colour data. The colour planes are all the same size as the full frame. The RGB data was created by transforming the orginal YUV data, but note that the reconstructions shown here were created by processing the YUV data. Note that the five frame reconstruction process used for large areas omits the first two frames from restoration. Thus one of the restored sequences (PULLRES2.SEQ) starts from the 3rd frame in the original sequence. There are two other colour sequences as follows:

The sequences illustrate that PULLRES2.SEQ gives sharper reconstruction. Nevertheless, the median result is quite pleasant. In both cases the improvement over the original sequence is appreciable. However, not all the distortion has been detected correctly and there is some distortion of moving areas. This is because there is considerable colour and luminance fluctuations in the film both temporally and spatially. Thus it is difficult to make a decision abou missing data based on intensity and there are many false alarms. In particular there appears to be a fine crack type structure in some frames. This is very difficult to detect as the contrast is very low. A global application of the vector median filter VML3Dex, after reconstructing the motion field using the five frame algorithm, may actually yield quite pleasing results here even though it would distort more of the motion in some of the areas.

Final Comments

This chapter has outlined the steps required for extending the techniques introduced in previous chapters to colour images. The reader will observe that the separate processing of each channel appeared to yield very usable results. Although it is clear that a proper multichannel approach to colour manipulation is potentially much more beneficial than separate channel processing, the improvements may only be observed at high levels of degradation. The further development of multichannel extensions to the systems presented in this book is currently being pursued.