All perfect praise be to Allah, The Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallahu ‘alayhi wa sallam, is His slave and Messenger.
Marriage is one of the great good deeds that yield many benefits in the worldly life and the Hereafter, such as increasing the human population and guarding one's chastity, and the like of benefits.
If this man is righteous, then our advice to you is to try to convince your family by seeking the help of some wise, virtuous people to mediate between you and them. If your parents agreed to the marriage, then this is the desired objective; otherwise, you are not obliged to obey them in this regard, let alone obeying anyone else other than your parents. Children are obliged to obey their parents in whatever is of benefit to the parents and, at the same time, does not cause harm to the children. There is no benefit for your parents in objecting to your marriage, and giving up marriage could cause you much harm. Please refer to fatwa 84942 about the limits of obedience to the parents. If you get married and your family or parents are angry about it, then try to use every possible means to earn their pleasure.
What they said about bringing a man into your daughter's life is a weak argument for which marriage should not be given up. Actually, this man's presence in the house may be in the best interest of your children as he would help you raise them properly. It should be noted that if the marriage is consummated, your daughter becomes one of his female Mahrams (permanently unmarriageable kin) because she would be his stepdaughter. Allah, The Exalted, says (what means): {Prohibited to you (for marriage) are ... and your step-daughters under your guardianship (born) of your wives unto whom you have gone in.} [Quran 4:23]
Allah knows best.