what he said.
Something isnt aligned right....either the carrier in the swing arm, the bearings in the carrier, or the swingarm in the frame. Possibly the wrong bearings are in there?
I'd doubt that it has anything to do with the sprocket or the chain....unless you have more play on the top of the chain than the bottom....maybe its one tooth off and not even(thats highly unlikely, but you never know)