I must admit that I like to ride close on a faster buddy's wheel down hill. That being said it's good if you develop that relationship a bit first. The same goes for when someone is following me close. If its a person I am comfortable with then fine but if not then it wigs me out a bit. One time I had a guy riding right behind me that I didn't know well and he skidded all the time. I kept thinking he was going to hit me so I let him in front and he wasn't actually that fast when he had to lead.
The good thing about stopping once in a while is that it lets the Natural order sort it's self out. In my experience if one person thinks they are holding another up then they offer the faster rider the next spot up the line.