My '03 shee was doing the same thing. It shifted fine slowly through all the gears, but when shifting fast there was all kinds of trouble. I jacked with the shift shaft and was never able to get the thing right. Finally i just purchased a new shift shaft, like $40.00, and all my problems went away.
Not sure if this link will work for ya,
http://216.37.204.206/wps/Yamaha_OEM/Yamah...e=13&A=257&B=15