brushing is pointless unless you're racing.
a quick couples laps down the hill on your fresh wax will achieve the exact result as brushing by hand.
brushing only affects the wax on top of the base, not the base itself.
you are correct in the rest though. if the skis don't cool at room temp (e.g. outside) the wax won't stay in the base properly. it's also easier to scrape skis when they're warmer.
This is incorrect. If you don't brush your skis they will be without texture until the wax wears off, and even still the wax will remain longer in the bases textured grooves, creating a essentially flat surface unable to effectively channel water. Obviously a faster base is more important in racing, but simply skiing will not recreate the same base structure.
Furthermore using a hard brush does affect and clean the base structure. Especially if you do base repair, this is the only way to add structure to the new base.
Take a quick read of the KUU tuning manual here under structure and brushes.