I think for me the main reason why I despise B/X is for the same reason why I hated even the idea of Dawn/Spike, is because I love family relations between chars. Big brother/little sister interests me as much, if not more than romantic relationships. Xander's interaction with Buffy didn't work as romantic because they lacked any chemistry and were boring together as a couple. But as a brother/sister thing they worked great because it showed that Buffy could be friends with a guy, that not every male she met had to be a romantic interest.

Xander's build up into being that confidant and big brother to me gives Xander a much greater meaning in the series than he could ever have had as a love interest.