All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
We do not find in what you said anything that is blasphemous against Islam or that you did something sinful. It is wise not to respond to a mocker or rebellious person. This is something to which Allaah advised His Prophet to do; Allaah says (what means): {And if they dispute with you, then say, “Allaah is most knowing of what you do.}[Quran 22:68] Al-Qurtubi said in his Tafseer (interpretation of the Quran): “If they put forth false arguments, then respond by saying, 'Allaah is most knowing of what you do,' that is, of your disbelief and denial. Thus, Allaah commanded him to avoid debating with them to keep him from getting caught up in their obstinacy. There is no response to obstinate people.”
However, if you can look for another job, then do so as this is better, in order that that man will not affect you or do something that will harm you. If you are working for him as a personal assistant, then you are obliged to leave this job as some scholars stated that it is forbidden for a Muslim to work as a personal aide for a non-Muslim as this involves humiliation for the Muslim, and we have already clarified this in Fatwa 81686.
Allaah Knows best.