Here is a cut and paste from a reply i posted a ways back…
The number of components per channel on MiniDV footage is 4:1:1 - which is bad news for Green and Blue screen keying, which is why it is so difficult to properly key DV. (4:1:1 refers to RGB channels). By blurring the Green and Blue channels a little bit and recombining the channels into a new image (before sending through a keyer) you can achieve much better results, or at least get rid of some nasty edges much easier.
Included is a diagram…basically what is going on is this…
Original footage is sent to three layer operators, so now we have 3 layers in the comp. However- we send the original red channel into all 3 RGB channels of the new Red Layer. Then we send the original Green channel into all 3 RGB channels of the new Green Layer. Then, we send the original Blue channel into all 3 RGB channels of the new blue layer. Then, apply a blur (4 pixels or so) to only the Blue and Green layers. Then, recombine the 3 New layers into one. At first glance the resulting output image won’t look much different, but Ultimatte or another keyer has a much easier time with it, and you won’t get near the aliasing around the edges. I tried it on several pieces of footage with varying degrees of improvement, but it’s worth a shot if you find something difficult to key. It also works better if you apply the blur in YUV space instead of RGB.
Sorry for the complicated diagram. I had to black out the thumbnails due to NDAs