man! this sounds like a very particular issue…
that said… what does happen when you import it into MB? i assume you can see the bones in motion… just having issue with retargeting?
ideally you would create your own Control Rig that operates correctly first…then choose your mocap data as a Source in the Character Controls
if it doesn’t appear in the Source pull down then it might not be compatible. Not all mocap files are equal regardless of format.
have you tried exporting it out of maya as an FBX?
also naming convention of the mocap skeleton can sometimes be an issue as well. e.g. Skeleton:LeftArm
if that’s the case “i think” there might be a way to rename the individual bones correctly in it’s character definitions… but i can’t be positive about that