The big difference for me is foot speed while pedaling. I'm 6'2", used to ride 175s, and have happily switched to 170s. On the 170s, my feet trace a smaller circle, so for the same cadence, my feet are traveling slower. Which means I can get a slightly higher cadence for the speed at which my muscles/knees feel comfortable, which means I'm getting the same power as before, with less torque, which means my legs just last longer. One thing I've noticed is that people tend to get out of the saddle to climb/accelerate way sooner tend to be on crank arms that are too long. They're either too far behind the pedals to push hard, or can't get the power they need whole seated without getting to an uncomfortable foot speed.
Furthermore, shorter cranks move the point where you put power down in the pedals toward the back of the bike, relative to where your saddle is, which effectively steepens your seat tube angle. This, to me at least, is a huge benefit on BC's steep climbs, where being in position to sit balanced and push on the pedals on a sustained climb is important. I can handle 175mm cranks on bikes with >75° STA, but I'd still rather not.