Wow, thanks all for your comments in the thread.
I will answer a couple of things pointed out to me from my original post.
Yes! is the simple answer, riding with my 18 month old is pretty much fun for me. I have never had an issue with people passing me when I am with him. He loves it too. Bump bump and cheering and laughing the entire way down and high fiving and fist bumping all the way up.
My issue was with my 4 year old getting passed. I ride behind her to give her a buffer and I make sure people know they are about to pass a 4 year old. But, please just do so with care. I do my best to ride at times that are not too busy, and trust me she can handle everything that bobsled can throw at her she just does it at 4 year old pace. All I ask is let us clear the track for you when it is safe and then get back to your top speeds.
Both my kids will learn proper trail etiquette as they grow up, including the climbing rider has the right of way and pass only when it is safe to do so. As their skills improve they will be looking to get on to bigger and better trails so this debate will never really end. it all comes down to RESPECT. Then we can all get along on the giant playgrounds that we are all so fortunate to get to play on.