divorce with TD status

My husband has a 3 year TN, and our eldest child and I have TD status. Our younger child is a USC. I want to leave my husband and move back to Canada, but I'm concerned that he could prevent me from doing so.

I know that I will need a letter from him, stating that he consents to my taking the children across the border without him. I'm concerned that if I tell him that I'm leaving him, he will not give me his consent and I will be forced to stay in the US, because I'm not leaving our children behind.

Does anyone know if I can be forced to remain in the US? Can he be legally compelled to allow me to take our children back to Canada? If I remain in the US, I will be financially dependent on him until he moves back to Canada, which would mean I'd be moving back home too. I don't have the education or skills to apply for a TN of my own.

Additionally, if I file for divorce here, USCIS will tell me to leave the country, since I cannot stay as TD if I'm not married to him. I was told this by an attorney here. I do not want to remain in an unhappy and emotionally abusive marriage, and I don't want to leave the children with their father. He's not physically abusive of them, but he's verbally abusive of all of us.