Well, the sign is pretty clear: bikes yield to horses and hikers. If there's a sign than there are rules in place. Since this is the only kind of sign that addresses the right-of-way issue in a multi-use area (including trails that are primairly MTB) than it seeps pretty clear that the hikers and horses have right-of-way.
Why?
Horses spook easily and have a person perched precariously on top of them. It's safer for everyone if the bike yields.
Hikers can get hurt if a cyclist hits them at speed. It's safer if they have the right-of-way.
Is this always the best arrangement for us? I dunno, but when I come across a car-sized object with a mind of it's own, I respect the damage that it can do to me and the person sitting on top of it. Also, I know how unpleasant/scary it is to have riders blow by me at speed while hiking as I have no idea how much control they have. If I bear this in mind I don't get so pissy about having my flow "ruined" by waiting for a hiker to go by.
Personally, I feel that a lot of the issues that we have with other trail users could be overcome by being friendly and communicating with them and by respecting their concerns. Mountain bikes approach quietly and quickly and make a big racket while going by. Horses can startle, hiking is a quiet, peaceful activity where one can really take in the surroundings, biking often requires keeping the pace up to clean a section. By bearing this in mind while out on the trail (on bike or in my hiking boots) I've made all kinds of friends amongst all trail user groups.
I sure wouldn't hike on a busy trail that's used and designed for mountainbikes though. Fairly unpleasent, and my presence can ruin the flow for them if I don't jump off the trail in time.
Just one trail users view, and one that's kept me conflict and frustration free for years.