While the women have the main right to their say as this is something that they cannot walk out of, I feel like (provided the situation is between open and willing partners and is not messed up beyond all fuck) the man's opinion can be taken into consideration. However, the woman is risking her health and her livelihood if she even tries to carry the child to term. Forcing someone to carry a child, given the bodily risks that they face alone, could be considered somewhere along the lines of domestic abuse when you really think about it, especially if the woman in question has a history of pregnancy problems or family members with pregnancy problems that could develop.
There is, of course, a way to sign away parental rights so if, in this situation, it's just the baby that presents a problem and not the pregnancy itself (which, let's be real, might be part of the problem because of how many health issues there are out there that can come up regarding this), then either parent could basically sign a waiver saying: "I give up ALL of my biological parental rights to this child and forfeit them to their mother/father." This actually can be done from both sides of the coin, so the mother or father could waive all parental rights. Again, this assumes that the pregnancy alone is not the issue... And there's a small issue of what to do if these contracts are not honored even after being drawn up (admittedly, this is a greater concern if the father agrees to take the child, signs waivers, and backs out at the last minute since that's easier to do on his side of the coin). Still, however, it is as thing you can do and a mother stuck with that child could still essentially put the child up for adoption.
Ultimately, though, this is something people should be talking about if they're in relationships with other people. It's a hard and fast issue and it should be one of those that most adults are aware of their stance. If you, as an adult, are ready and want kids, be prepared to look for/date/marry people who are ready and want kids. If someone is not ready and does not want kids (or just is not ready or just doesn't want kids) pops up on your radar, it is your responsibility to take charge and say, "You're cute and all, but this is a major issue we can't reconcile, so this can't happen." Sex is gonna happen, sure, but it's part of the responsibility of each and every pair of adults who can have kids to understand that they need to see eye to eye on this subject. There are tons of people in the world, so making that a point of contention with any relationship (regardless of where you stand) is just a way to make sure that you as an adult are pursuing the life you want.
I'm not saying "don't have sex" with that comment either. I'm saying "whether they want kids or not should be in your requirements for whether you have sex with that person specifically." Of course, this doesn't branch into rape cases and, of course, becomes very complicated when in concerns to one night stands, but at least having this being a more commonly accepted approach might help immensely.