I think your problem there is going to be what it should look like where you’ve got particles overlapping, or say, an arm of a character overlapping the body.
I just did a quick test using the ambient occlusion shader from mental ray in the opacity slot of a standard material or the cutout slot of an Arch Design material and it looks promising.
I stuck a noise map with negative output value in the dark slot of the a/o shader to give it a more definite falloff to nothing, but it looks like it could work once you’ve got the right amount of falloff/spread.
I’ve got to nip out now, but I might have a proper play with it tomorrow, see if I can get it working in a real world example.