With the help of one of the other Technical Artist at work I now have been enlightened on the matter and I thought I would share.
When you make a setDriven key in Maya it acturally creates a new node (per channel) inbetween your driven and driver attributes. In my example I just made two cubes and set one cubes translateX to drive the translateX on the second cube. If you querry the input and output connections in HG you will see the new node inbetween.
This new node is an AnimationCurve. There are 7 types of animation curves from what I was told. So, to get rid of them, all you really have to do, is querry the connections of your object via listConnections. The walk though the connections and verify that they are animation curves via the handy isAnimCurve function and then delete the node if true.
Here's the sudo-code version.
// list your connection from your object, put in var
// verify that the input connection is a AnimationCurve
// delete :)
Hope that helps someone out there. I didn't see any write ups on this matter. Most threads are about creating, not deleting.