One of the strange things about reviewing equipment is that, occasionally, you stumble across industry-wide problems, and initially doubt your sanity. How could all these fine consumer electronics manufacturers get things so wrong?

For example, when DVD Audio first came out, it took me a little while, but then I discovered that no DVD Audio players provided either bass management or speaker time alignment, so except for a very few people with high end setups and equal distances to all their loudspeakers, DVD Audio was incapable of producing the correct sound field. Thankfully this has largely been overcome, although I note that SACD players still don't provide time alignment except on a few players that convert it to PCM.

Previously I have set out my doubts about the value of progressive scan for PAL DVDs. That argument was essentially negative: progressive scan doesn't do anything much for PAL, so why bother?

But in the past few days I have discovered something far worse: with many DVD players progressive scan reduces picture quality on the vast majority of PAL DVD movies. That has a huge impact on the value of HDMI. Let's see why.

Film and Video sourced DVDs

Our TV system is interlaced. Let me recap briefly what that means. I shall only talk about PAL for the time being.

A standard PAL TV shows 25 frames per second on the screen. For DVDs, each frame is made up of 576 viewable lines.

If our TV system was progressive, it would work like this. An electron beam inside the picture tube 'paints' the picture on the phosphor coating on the end of the tube. It starts at one corner, zooms horizontally across to the other corner, switches off for an instant while the aiming point zooms back to a point just below the first starting point, then it repeats the process. When it finally gets down the bottom of the tube, it switches off for a longer period then comes back to the starting point.

But for interlaced signals, the process is a little more complicated. When the electron beam finishes each line, it doesn't go down one position but two. Thus it paints the first line, then the third, then the fifth and so on, down to the bottom of the screen. For PAL, it reaches the bottom of the screen in one fiftieth of a second. Then it flips back up to just below the top line, and paints the second, fourth, sixth and so on lines down to the bottom. This takes another fiftieth of a second.

The first top-to-bottom scan is called a field, as is the second. Each field takes one fiftieth of a second. Two fields together make a frame, which takes one twenty-fifth of a second.

Given the technological limitations when TV was first introduced, interlacing made sense. The alternative would have been to use progressive scan at 25 frames per second. The whole picture would have been drawn in one sweep, then the next picture would start one twenty-fifth of a second later. But between each frame, the phosphor would have darkened and whole-screen flicker would have made the picture unwatchable.

The main difference between film and video sourced DVDs is that the first are inherently progressive, whereas the second are inherently interlaced.

Most video cameras capture the image in the same interlaced way as old-fashioned TVs display them. They first capture every second line to create a field, then one fiftieth of a second later they capture the in-between lines to create the other field. But whatever is being videoed will most likely have moved in that one fiftieth of a second. So the two fields, when reconstituted into a frame using a simple 'weave' process, will be out of alignment. That means that on a modern digital display, which is inherently progressive in nature, you will see jagged horizontal lines standing out on either side of moving characters. This I call 'combing'.

Figure 1 shows an example of this. It is a detail from a PAL DVD, 'The Gunston Tapes', which was originally an Australian television production. The hand you see in the picture is that of former Australian Prime Minister John Gorton being extended, from right to left, towards Norman Gunston. The bottom of the three is the original screen capture, but cropped to 100 by 100 pixels and then doubled in both dimensions to make the combing clearly visible.

This bottom frame is constituted by two fields, which I have extracted from the original frame and placed above it. One fiftieth of a second elapsed between the first and second fields, which is why they are displaced with respect to each other. The 'weaving' process simply interleaves the two fields together and produces the jagged result you see at the bottom of the graphic. In fact, in order to capture that frame, I forced CyberShot PowerDVD into 'weave' deinterlacing mode.

In order to produce a smooth picture, a more complicated deinterlacing strategy is required. How that works I'll get to in a moment. But first, let us briefly consider DVDs created from films. A film frame is designed to be shown in one 24th of a second. A PAL video frame is shown in one 25th of a second. What the hell, this is close enough. There is a direct one-to-one transfer of each film frame to each DVD video frame, and the whole movie proceeds at a slightly faster clip (roughly 4% faster in fact).

But in creating each video frame, the film first has to be put into interlaced format for compatibility. So each film frame is displayed in a telecine (a device for transferring film to video), and then the two fields are captured. One field has all the odd scan lines, while the other has all the even ones.

Since it is the same film frame in both fields, there is no horizontal displacement between them. In order to show a nice picture on a plasma display, all the TV has to do is 'weave' them back together. That's pretty simple.

But let us not talk about TVs. Let us talk about DVD players. Let us assume that you have a DVD player that will convert the video to progressive scan format which, as everyone knows, produces a better quality picture than an interlaced video connection. So your progressive scan DVD player ought to take PAL DVDs with film sourced content and simply weave them together. As we shall see below, this does not happen. But first, let as see how video-source material is de-interlaced.

Deinterlacing video-sourced content

Now, if having severe combing on your TV or projector display is to be avoided, what must your DVD player do in its progressive scan conversion? There are essentially two deinterlacing techniques that may be employed. And both are fudges to make the best of a bad situation.

The ideal way, I suppose, would be for the video processing circuitry to examine the combing lines, work out where they belong according to the alternate scan lines, and slide them back into place. But in fact that is practically impossible, even if you had enough computing power to do it. Consider Figure 1 again. Look at the dark text on the wall above the hand. In this scene, the camera is panning from right to left, the hand is extended also from right to left, but faster than the camera is panning, so in the second field it is further to the left than in the first field. But since the camera is panning from right to left, the fixed text in the background is, from the frame's viewpoint, moving from left to right. So its combing lines are extending to the right. So which lines would the processor slide? What would it put in the gaps left behind? It just can't be done.

The most common technique for deinterlacing video is called 'bobbing'. Depending on which explanation you read, there are two ways in which this can be done. I shall ignore the simpler one (which involved simply eliminating the first field) and discuss the only the second.

In bobbing, instead of showing both fields at the same time, they are shown one after the other, each for one fiftieth of a second. Each field, displayed on its own, would leave virtual blank scan lines during its instant of display. Even if they were thin enough not to be noticed (although they would be noticed), this would halve the effective brightness of the screen. Something has to be done about that. The cheap and nasty way is to simply double each line. That would result in a noticable step pattern on diagonal and curved edges, so this is rarely done. Instead each 'blank' scan line is filled in with a new line, created by interpolating between the line above and the line below.

There are three main results of bobbing:

1. Reduced effective vertical resolution. Sure, everything is shown, but each field is displayed so quickly, and its details are partially overwritten by the persistence of vision in our own eyes and brain, that we don't see it all. That's not the only problem.
2. Broken detail. With fine, high contrast detail our persistence of vision isn't sufficient to smooth out the differences between the two fields, so some details can appear with jagged steps, especially viewed with a large display like a front projector. This is most obvious with the scrolling credits at the end of many movies, where the small white characters have flickering black lines across them.
3. Moire. A moire pattern is a kind of interference pattern. Put one piece of fly screen in front of another, look through them both at something beyond, and slide one across the other. The interference of the two fly screens will produce broader bands of pattern. Sometimes bobbing can cause this on fine regular patterns in the video, such as clothing patterns.

The second and third results cannot be readily demonstrated with stills because they are produced by the alternating fields. But the first one can be. Look at Figure 2. This is a detail (at original size) of the shoulder of a man in a clip I captured. The source is film based. For the top part of the figure, weave interlacing has been applied while for the bottom, bobbing has been used. Notice the fine near-horizontal lines on the suit jacket? There's a big difference between the two.

The top band of lines on this suit jacket, which are pretty much invisible in the bobbed still, also appear with a marked moire pattern when the clip is shown with bob deinterlacing, but it is clean and accurate with weave deinterlacing.

But there is a better way. It is motion adaptive deinterlacing, as seen in Faroudja DCDi and HQV processing. These systems analyse the picture and still apply bob deinterlacing, but only to those parts of the picture where combing would otherwise be present. Often parts of the picture are steady, while other parts are moving, so full detail is preserved in the static parts of the picture.

The Problem with PAL DVDs

But how is a DVD player to know whether the picture is from a film or video source? The official DVD Video specification provides for this. Within the data stream of the DVD Video are flags for indicating various attributes of the video. One such is the aspect ratio flag, which can indicate 4:3 or 16:9 (there are actually other options as well). This flag allows the DVD player to correctly format the picture for 4:3 displays.

But the one we are concerned with here is called 'Scan Type'. This can be flagged in one of several ways, and can in fact switched between the different settings as the movie proceeds. But there are two settings that most commonly appear on DVDs: 'interlaced' or 'progressive' (the actual flag is simply a couple of digital bits, set to some value, which the DVD player can interpret, not these actual words). There is another way, used by some higher end DVD players, that avoids this whole problem. These players actually analyse the incoming video signal itself and determine from that the appropriate deinterlacing strategy to use. See Update 2 below.

Now if you take a decent NTSC movie, which obviously originates from a film source, that flag will generally carry the value 'progressive'. The DVD player reads this and knows that to generate progressive scan output, it simply has to weave the fields together to create frames. The DVD player also knows that it has to perform 3:2 pulldown. But if you have an NTSC TV show that was 'filmed' with video cameras, the flag carries a value of 'interlaced'. The DVD player knows then that it should not perform 3:2 pulldown, and it has to apply a deinterlacing strategy appropriate for video sourced material in its progressive scan conversion.

Now let's forget about NTSC and consider PAL DVDs. I was startled when I started looking through my DVDs the other day to discover that the flag on every single movie DVD I checked was not 'progressive', but 'interlaced'. I do have one PAL DVD with the video flagged 'progressive'. It is Digital Video Essentials, a test and setup disc, and it demonstrates that there is no technical reason why PAL DVDs can't be appropriately flagged. So why aren't they, when their sources are clearly progressive?

I contacted a couple of Australian DVD Authoring houses to find out. Both Paul Rodger from Stream AV and Peter Campbell from IML Digital Media tell me that this is part of the official DVD specification [see below for update]: PAL DVDs, even if film sourced, have to be flagged 'interlaced'. Why would this be? I suspect that it's because a 'progressive' flag on NTSC DVDs not only tells the player which deinterlacing strategy to use, it also causes it to invoke 3:2 pulldown, which is not appropriate to PAL DVDs. Anyway, the reason is unimportant. What is important is the effect this has.

And that effect is that the DVD player reads the flag on a PAL DVD (which, remember, says 'interlaced'), consequently thinks that the DVD is video-sourced, and so invokes whatever deinterlacing strategy it uses for video material. In the case of most inexpensive DVD players, that strategy is to bob the video.

Implications for HDMI

HDMI video from DVD players is in progressive scan format (exception: the new Pioneer DV-490V can output 576i). It pretty much has to be because a lot, maybe even most, displays will not accept 576i via HDMI input. HDMI isn't actually specified to support 576i, only 480p, 576p and higher.

So if you use a HDMI connection to your display, your DVD player will be deinterlacing film-sourced movies (which is most DVDs) using an inappropriate strategy designed to deal with video-sourced content. As a result the picture quality is damaged.

Some HDMI DVD players (for example, the Panasonic DVD-S52 I'm using at the moment) have a wonderful option: you can override the automatic scan type detection and force the player into either 'Video' or 'Film/Cinema' mode (the name varies). With the Panasonic you have to change this (painfully, because it's several levels down through the menu system) every time you insert a DVD because it defaults to 'Auto'. But many DVD players, such as the Philips DVP-5965K, don't have such an option. It will always play back your PAL DVDs using bob deinterlacing, and your movies will therefore suffer from soft focus, broken fine detail and occasional moire patterns.

Oh don't be fooled by the prospect of switching the HDMI output of your DVD player to 720p or 1080i. In both cases the video goes through deinterlacing first before being scaled up, so there is no effective improvement using either of these.

Conclusion

What all this boils down to is that progressive scan is broken in the PAL DVD world. Badly. Even if your DVD player has, say, Faroudja DCDi deinterlacing, this is merely less bad than bob deinterlacing, and totally unnecessary.

If your DVD player has no option for forcing film mode deinterlacing, you will actually get the best picture quality by eschewing the HDMI output. Instead, use the analogue component video outputs and make sure you switch off progressive scan. A good quality display ought to detect that the incoming fields are lined up and therefore simply weave them together. Some projectors and other displays even have an option to allow you to force the video to be treated as film-sourced, ensuring the deinterlacing is performed using the weave method.

Don't get me wrong. I love HDMI. It provides the best picture quality currently available from DVD. But only if you can make it treat your DVDs correctly. If you are in the market for a HDMI DVD player, just make sure it has a 'Film' or 'Cinema' mode display option.

© 2006 by Stephen Dawson

UPDATE (20 August 2006 -- return to text): I emailed the DVD Forum regarding the claim that it is a requirement of the DVD specification that PAL DVDs have their scan mode flagged as 'interlaced'. However the DVD Forum denies that this is in fact a requirement. Here is their reply:

DVD-Video Specifications do not require flagging for interlaced or progressive. It might be required in MPEG specifications, but it is up to software producers to choose which mode is appropriate for their product.

UPDATE 2 (28 October 2006 -- return to text): Californian DVD maker, Oppo Digital, has written and reminded me that some DVD players use cadence detection to determine whether to weave the video or employ a deinterlacing strategy appropriate for interlaced video. The reason for this is that there has been a long history of incorrect flagging in DVD video streams, including on US discs. If implemented properly, this should avoid this whole issue. It is worth noting that Oppo Digital's own DVD player scored brilliantly on this, and most other, aspects of performance under the rigorous testing procedures at the Secrets of Home Theater and High Fidelity web site (big table, takes a while to load).

Here is Oppo Digital's explanation:

The most common form of de-interlacing DVD player is a flag reading DVD player. A flag reading DVD player will rely completely on the flags embedded in the MPEG stream to do the de-interlacing. The problem with this form of de-interlacing is that films generally break in and out of flags, and any interruption of the playback (such as chapter skips or even chapter breaks) will cause the unit to lose the flag. This greatly decreases the overall picture quality.

A cadence reading DVD player is one that analyzes the frame content straight from the MPEG decoder to figure out what de-interlacing algorithms should be used. Flags are used as markers rather than discrete instructions. However, interruptions in the cadence (such as a switch to 2:2 Film or Video) can cause the unit to stay in an improper Film mode. A good cadence reading DVD player does not suffer from this error.