All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger.
First of all, it should be mentioned that since the father of your two children is a non-Muslim, then he has no right to foster them, because among the conditions for a person to be a fosterer is that he/she should be a Muslim, as stated by the jurists . So, if these two children are living with him, you must endeavor your utmost to get them away from him so that he would not corrupt them.
As regards the treatment of your current husband to these two children, then it should be with a good manner while being cheerful with them and by him gifting them some simple gifts that do not burden him. In fact, this is a kind of calling people to Islam and it is being kind to the wife as well; all this is Islamically urged and encouraged. So, he should show them the beauties of Islam and he should encourage them to be brought up according to (the teachings of) Islam.
On the other hand, you should have a mutual understanding with your husband about this matter in a soft and gentle manner and clarify to him its importance for your two children and that they need that and appreciate it. Also, you should remind him to seek the reward of the Hereafter in doing so. You may show him some texts about the excellence of calling people to Allaah and that if a person is guided to do a good deed thanks to someone, then the latter gets the same reward as the person who did that good deed. Moreover, actions speak louder than words in calling people to Allaah.
In addition to this, you may try to get them involved in a good company or a good environment through the activities carried out by some Islamic Centres for the children of their age group as in general this will have a great effect on them.
Finally, you should not forget to supplicate to Allaah as much as possible to guide them and rectify them as this is the practice of the Prophets and the righteous people; Allaah Says (what means): {And those who say, “Our Lord, grant us from among our wives and offspring comfort to our eyes and make us a leader [i.e. example] for the righteous.”}[Quran 25:74]
Allaah Knows best.