Smart Deinterlacing Filter for VirtualDub
Version 2.2

[Place this text file in the VirtualDub plugins directory to make
it available via the Help button on the filter configuration dialog
box. The computer must have a program called Notepad available in
its search path.]

This filter provides a smart, motion-based deinterlacing capability. In static
picture areas, interlacing artifacts do not appear, so data from both fields is
used to provide full detail. In moving areas, deinterlacing is performed.

The following options are available:

Motion Processing

Frame-only differencing: When this option is checked (default), only inter-frame
comparisons are made to detect motion. If a pixel differs from the corresponding
pixel in the previous frame, the pixel is considered to be moving.

Field-only differencing: When this option is checked , only inter-field
comparisons are made to detect motion. If a pixel differs from the corresponding
pixels in the previous and following fields (the lines above and below the current
line), the pixel is considered to be moving.

Frame-and-field differencing: When this option is checked, both inter-frame and
inter-field comparisons are made to detect motion. If a pixel differs from the
corresponding pixel in both the previous field and the previous frame, the pixel
is considered to be moving.

The correct choice for differencing depends on the input video; each mode has problems
with some kinds of clips. Field-only differencing alone may tend to overestimate motion.
Frame-only differencing may tend to underestimate it. Frame-and-field differencing
attempts to compromise. Also to be considered is execution time; frame-only will be the
fastest, followed by field-only, followed by frame-and-field.

In my experience, good results with a wide range of clips are obtained with frame-only
differencing at a threshold of 20, with motion map denoising enabled, and so these are
the filter defaults. Motion map denoising is especially helpful with field-only differencing
as it reduces the amount of picture detail that is erroneously detected as motion.

Compare color channels instead of luma: When comparing pixels, if this box is unchecked
(default), the lumanance values are compared. When this box is checked, the individual
color channels (red, green, and blue) are compared. Luma comparison is good for general
video and especially where static alpha-blended logos appear (because video noise can
cause subtle color changes that would be detected by color channel compares). Color
channel comparisons are good for cartoons and other clips with large solid color areas.

Show motion areas only: When selected, only the moving areas of the image
are displayed; static areas are black. This option can be used to assess the
suitability of the choice of option settings and threshold.

Blend instead of interpolate in motion areas: If this checkbox is not selected (default),
then, in motion areas, the filter will discard one field's data and recreate it
by interpolating new lines from the retained field's lines. In static areas, both
field's data is used. If this checkbox is checked, then, in motion areas, instead
of interpolating, the filter blends each line with the lines above and below. This
has the effect of blending the fields, which tends to blur out the interlacing
artifacts. The choice of interpolation versus blending depends on the nature of
the input video and your own esthetic preferences. The interpolate mode avoids
the halos you can get with the blend mode, but it introduces some small amount
of stairstepping, and may tend to emphasize any video noise that is present. Try
both ways and see which you prefer for a given video clip.

Motion map denoising: A dilemma for users of the filter is that to get the best deinterlacing,
we like a low threshold, such as 10-15. A low threshold ensures that residual interlacing
artifacts don't sneak through. But if we set the threshold too low, video noise gets
detected as motion. This causes two undesirable results. First, the motion noise causes
a random sprinkling of deinterlaced areas in what should be static areas. This often
manifests as a kind of sparkling, which is objectionable. Second, any extra false
motion that is detected reduces the picture area that is passed through from both fields,
reducing the perceived resolution of the overall picture. So, what we want is a low threshold
but without the effects of video noise. When this checkbox is checked, extra
filtering is added in the motion detection pipeline (not in the main video pipeline, so the
ouput video is not compromised) that does a good job of suppressing false motion noise.
The downside is that the filter runs slower. Use the "Show motion areas only" option to
tweak the threshold fairly low without introducing false motion noise. This
option is especially helpful with field-only differencing.

Threshold: This value determines the difference between a pixel and its corresponding
value in the previous field or frame that must be exceeded for the pixel to be considered
moving. A threshold that is too high will allow interlacing artifacts to slip through. A
threshold that is too low will cause too much of the image to be treated as moving,
reducing the perceived resolution. A too low threshold will also tend to emphasize noise.
Without motion map denoising (see above), a threshold of 15-35 is good.
With denoising, 10-20 is good. You can view the effect of threshold on motion detection
by selecting the "Show motion areas only" checkbox.

Advanced Processing

Advanced Processing is almost always used for "deinterlacing" captured PAL video originally
made from films. If you have PAL movies that seem to be interlaced, it is worth trying
the advanced processing first, because it may allow you to recover the original progressive
frames without quality loss. Because the processing is sufficient to recover the original
progressive frames, full motion processing is disabled when Advanced Processing is enabled. 
I give the simple story first, then I explain in detail what is happening.

Simple Story

If your capture card uses field order A and the PAL video looks interlaced on your computer,
then check phase shift, uncheck input swap, and check output swap. If your capture card
uses field order B and the PAL video looks interlaced on your computer, then check phase shift,
check input swap, and uncheck output swap. If you are unsure of the order of your capture
card, then try it both ways.

Please note that, due to
the way that this filter buffers frames internally, it may appear to not work correctly when
scrubbing the time line or when single stepping backward. But single stepping forward will
always work and, of course, saved processed video will always be correct.

Here is a description of what is happening in detail.

Phase shift by one field: Consider a frame/field sequence of progressive video
AA BB CC DD EE... This is correct because each frame contains the correct field data
from the original progressive frame. Suppose, as can happen, the stream is
encoded for display on today's interlaced TVs, but the encoding is
off by one field. Then we receive ...A AB BC CD DE E... The picture will look fine on
the interlaced display, but it will appear to be strangely interlaced when
displayed progressively on a computer. Apparently, this is a common problem in the
PAL world. It can result from two possible ways that a field order mismatch can occur.
1) The video is telecined normally (T1B1 T2B2 T3B3) but the capture card uses
field order B, or 2) the video is telecined using a perverse ordering (T1B2 T2B3 T3B4),
which displays OK on the TV but is strange!, while the capture uses field order A. (The other
two combinations of telecine method versus capture order work out correctly;
reordering is not required.)

The phase shift option, when enabled, simply shifts the field phase by one,
that is, T1B1 T2B2 T3B3 becomes B1T2 B2T3 B3T4.
If neither of the swap boxes are checked, this pure phase shift is performed. This will not
be useful for PAL but may be useful in cases where a pure phase shift is desired.

...with input field swap: Modifies the phase-shift-by-one-field option by swapping the
fields of input frames prior to the shift. 

...with output field swap: Modifies the phase-shift-by-one-field option by swapping the
fields of output frames after the shift.

Very patient readers will be able to see how the settings described in the Simple Story
properly restore the progressive frames for PAL movies with mismatched fields in the
two mismatch scenarios. Impatient readers can trust me!

If the phase shift box is not checked, Advanced Processing is disabled and full motion
processing occurs.

Note on 3:2 Pulldown, Etc.

Some video streams are created from progressive video by a process called pulldown.
This results in a mixture of progressive and interlaced frames, from which it is theoretically
possible to recreate the original progressive frames without doing deinterlacing.
However, it is technically very challenging to write a filter that will recreate the original
frames for all types of pulldown and in the presence of video edits, etc. While my goal
is to eventually make available such processing via options, the technology is not yet
ready. In the meantime, the normal deinterlacing will function quite acceptably for the
majority of these video streams. 

Feedback and suggestions will be gratefully received.
For additional information, version updates, and other filters, please
go to the following web site:

Filters for VirtualDub
http://www.geocities.com/Colosseum/Pressbox/8699/index.html

Donald Graft
neuron2@home.com
August 23, 2000
(C) Copyright 2000, All Rights Reserved