I believe it also has a bit to do with whether the pollee has read any of the books. Connery was the closest to the original stories, whereas Roger Moore is more appropriately associated with the Saint.
Depending on one's age, the choice of who is the best James Bond will be based on who the first one seen is. I'm old enough that Sean Connery is the "only" JB; others (my daughters) think it is Roger Moore. Go figure!
Who voted for Timothy Dalton? He's not a RRRRRR-ANGER!! UNACCEPTABLE!
(See below video at about 1:40)