Interesting idea of making your own thermal epoxy. I wonder if the aluminum oxide or diamond would risk shorting coils.
The standard thing to use is something like this
http://www.epoxies.com/therm.htmProduct 50-3150FR is about as thick as chocolate syrup or ketchup (according to Wikipeida
http://en.wikipedia.org/wiki/Viscosity )
Product 50-3100 is about as thick as peanut butter
I think those are the best two products from that supplier, there must be more suppliers of products like that, those kind of epoxies are very common in the motor building business. It is important to get vacuum on the parts when you apply the epoxy, that way all the air is removed to ensure that there is good conduction within the coil itself. If there are air pockets in there you will have local hot spots that can be significantly hotter than the rest of the coil which could cause problems if you are pushing the limits of the motor.
to bobc .. the idea of using the conductor itself to remove the heat is attractive, but the length of the copper and small cross section of the leads makes it infeasible. There are large motors which actually have hollow copper wire, and they pump cooling liquid right through the center. I haven't found any hollow conductors that would be suitable for small applications.
-ryan
