Something you also have to remember is if the doctor says "Oh shit yeah. You'll be able to ride full tilt, bang 5 women at a time, and pull a loaded diesel with your teeth." then they might become liable if you somehow try, and injure yourself.
After I hurt my back lifting weights 10 years ago, every doctor and chiropractor I went to said "Oh, you'll never lift anything more than 25lbs again in your life."
I was lifting 2 weeks later. I still have some issues, but I'm smart about it. 600lb deadlifts are out, but I can still hike up 335 on a good day with little issue.
I'm not saying to go out of your way and contradict your doctors advise, but remember its still your body.