You know what man, as cliche as it sounds you can't let it bother ya. As a matter of fact I know what you talking about. Hell, my father said that woman in our family will never lead good lives (as in sisters, wives, etc) How you think that makes me feel??
Anyways back about you. If it feels like work then it's not right. Don't they say if you love your job if you'll never have to work a single day. Relationship = work, but if you love it enough you'll never feel like working.