I'm not convinced that is totally true. These people are experts...and yes with human faults...sometimes right sometimes wrong. MOST people they put on the field can play...even if they ain;t HoFers.
I been plying guitar for a LONG time. I can hand you a guitar and immediately tell if you can play or not, just by the way you receive the guitar and hold it. Strum a few chords and scales, I can most likely tell if you can actually play songs in a band. You wouldn't need to perform a rock concert for me to be able to do that.
Having said that....there could be a SLIGHT chance you go into bad case of stage fright in front of 10,000 people.
Regarding your question...sometimes you roll with your starter even if you suspect the other guy is pretty good. It's a toss up and there's always some random factors involved. Soemtimes you need forced to make that change.