first check the above.
Make sure you are fully on the idle circuit . What i mean is some times when the pilot is to big people will compensate and get it to run right by having the idle screw in to far to get it to idle..
what this does is when ya let out of the throttle it does not block all the air as it should resulting in the bike still pulling from the mains until the rpms get low enough for it to not pull from them. After it will return to the idle circuit but not 100%. In addition, with that size motor through the smaller carbs it will have a strong signal coming in, thus making it more of a fine line.
The fix would be a smaller pilot, lower the slides down using the idle screws. then start it, move both idle screws in equally till it ldles on its own. Then while running adjust the air screws until you get the highest idle while still returning from a rev without hanging rpm.. after lower the idle screws to the desired rpm for idle.
Make the same passes after doing this and see what happens.
how far in are your idle screws? this will be the first sign that it could be doing what is stated above.
Just my .02
Chris