I have stories like that, too. Had quite a dilemma recently when I found out why a particular story wasn't working for >50% of the people who read it (including all editors so far)... and it's the bit I like best about it. Finally decided, as you have, to keep holding out for that editor who likes lemon cake.

Go you for getting the feedback!