Think about all the people who said no to you. You don't cross them off your list. You still respect or love them. And if they are grudge holders, do you want them in your life anyway?
They've been taught since childhood to be more docile while boys tend to be more rebellious and rambunctious. We're always wanting to make people happy.
In reality, the fallout is never as drastic as we think it is. People move on. Once you say no, people move on to find somebody else to do the deed for whatever they need done.
It will be someone in your inner circle, people you really care about. It's going to be something you have time for. It's going to be something you're not going to resent doing. It's going to be something you don't feel duped into doing. Just stop and analyze what's being said.
These people aren't thinking about you. What they are thinking about is who they can get to do this for them.