Last night I started working on a blog entry on this subject, but soon after I started I realised that I was putting in too much work to just give it away. So this is just a brief taste. Full details will be in a forthcoming issue of Sound and Image magazine.
The picture to the right consists of details from two closeup photos I took last night of a 1080p LCD display. In both cases I was feeding a 1080p test signal from a DVDO iScan VP50 to the display. The test signal was alternating black and white lines for the entire 1080 line height of the picture.
The picture on the left shows the excellent result achieved when I enabled 1:1 pixel mapping in the display. The picture on the right was with the default setting of the display, which scaled up the picture slightly. Obviously one is far more accurate than the other.
In the article I have used a analogy of the ‘beat’, the cyclic volume variation when two notes are played which are very close to, but not quite exactly, matched. That led to a discussion with my editor of the differences and similarities between the auditory and visual worlds. Naturally I had some thoughts on the matter.
Both musical beats and the display example in the photos just come down to interference patterns. You could make the reverse analogy and say that a beat generated by two slightly mismatched tones is actually an aural moire pattern.
However in the visual field the effect is far less obvious than the musical field. It’s amazing how much rescaling and conversion of video that can take place with remarkably little impact on the picture. Try to do to music the stuff that we do to video, and it would sound like crap.
I think its because of the differences between the ways our eyes and ears work. The latter do divide sound up into several frequency ranges, with different parts of the ear responding to different bands, but in the end it is all woven together and we get a seamless capture of up to ten octaves.
Our eyes only have a range of one octave, so that immediately rules out doing anything artistic with light in a way akin to music. It’s the coincidence of harmonics that makes musical harmony work: the second harmonic of C is almost identical to the first harmonic of G — there is just 0.89 hertz difference between them for Middle C and the G above it, and if you listen carefully you can hear the slow beat; and there is 1.18 hertz difference between the third harmonic of Middle C and the second harmonic of the F above it.
The other difference is that our eyes treat brightness and colour very differently. It is colour, of course, that is the equivalent of tone in sound. But we do not see a continuous spectrum of colour in the way that we hear a continuous spectrum of sounds. There are three colour bands, and all the light frequencies near to the centre point of one of those bands are lumped together and captured by just one of the three types of pigmented colour receptors in the eye.
What our brains tell us is a smooth range of colours is actually the results of mixing three levels of response from the three types of receptors to three bands of frequencies. In fact, you can take two sets of three frequencies, and if the levels are adjusted correctly, you can make them both look the same, although they would measure very differently with a suitable instrument.