Here are some pics man.
Pull the slides out and stuff a rag all the way through to catch the shavings. Then blow them out with compressed air afterwards!
You can use the grinding burr as shown, or a file or what ever you have, just take it down some so there is still some meat left for the nut to seat on.
Take your time, there is no hurry.
Your suppose to do this before you drill and tap the holes ( i know you didn't install them) so you "might" have to clean up the threads lightly after. Its pretty soft aluminum so its easy.