I first would have a serious (sit down) talk with your husband on how you're feeling. Emotions can definitely run high when pregnant. I would also suggest to your husband to see a Dr if he keeps getting sick like that. Whether it be anxiety or possibly another underlying condition or illness. I just lost mine to cancer this past July. He was the healthiest man I ever knew until he suddenly became ill. He was living with Renal Cell Carcinoma (Kidney Cancer) without knowing it until it metastasized (spread) to his lungs, then eventually taking over other parts of his body. He had NO symptoms at all until it spread. He/We exhausted every type of procedure possible for him to live. He desperately wanted to live, to stay. Unfortunately, none of the treatments or procedures worked, he suffered and died 6 months later. My reason for telling you this is because life is unpredictable, and without a doubt makes a habit of it. Therefore, I would have that serious talk with your husband on how you're feeling, and encourage him to see a Dr., as said. There's no wrong way here, nor is there no wrong feelings. Don't make any rash decisions, just yet. Talk to him and/or with someone that can help, together. Once you have talked it over, trust in your love and instincts as you both go forward, your answer will lay there.
My best to you!