Controllers to heat up, but like motors, to varying degrees depending on the capacity, condition, usage. Most are mounted within an aluminum chassis that may or may not have heat dissipating fins to help. I have a fairly big controller mounted within a rigid tail box on my trike, and found that I needed to ventilate it and even added a silent fan to help with air flow, as we are experiencing near 100-degree temps where I live. You will find some conversion kits that are designed so the controller is contained in an enclosed pack or bag - not optimal for cooling of course.
These controllers are obviously intended to run in the open air stream, although some are better sealed against the elements. I'd estimate that if I push my controller hard in the heat of the day, the external case reached nearly 150f - hot enough you can touch but not really hold your hand on there long. Long term heatup can certainly stress the components and lead to failure. If you give them adequate airflow and don't over-push them, they should have no problem. I have only had experience with a few controllers, all in the 48v 25-40A category. There certainly must be some that run cooler than others. I think motor overheating is potentially a bigger issue, at least in part because they are costlier than controllers, in general.
I should mention, as a beginner you will find motors, especially mid-drives, where the controller is contained within the motor or drive housing, or other controllers that are small and can be contained within the battery pack etc. Most of the stories you read here around controller failure are only occasionally related to overheating, and even then, often brought on by aggressive current application. (which is fun but costly!)