All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
The scholars who believe that the congregational prayer is obligatory state that this obligation is annulled by any legal excuses. Their evidence is the Hadeeth reported by Abu Daawood and Ibn Maajah that the Prophet said: “Whoever hears the Athaan and does not come (to the congregational prayer) his prayer is invalid unless he has a sound excuse.” The scholars defined the excuse that permits the non-participation in congregational prayers as the fear of almost certain harm or injury.
If your father's fear is of this kind, you should abide by his order, especially if your doing so encourages your family to continue performing prayers at their fixed times.
Allah knows best.