Only that kind of person would say that sort of thing, I think? Because there's some unspoken implication that you don't think it'd be easy for someone to care about you because of whatever flaws you might not have.