My mother also has GBM stage IV. I miss seeing my grandchildren like I used to. Unfortunately the noise and the activity of the kids (3 and 1 yrs old.) I also have used those exact words to my daughter when I see it is being affected by them being there so long. A half hour is about all any of us can take. I am exhausted from caring for her for the last 6 months. She hasn't been able to walk or talk since Thanksgiving. Tell your husband, his mother is sick. He should be the one with understanding. It is a terrible disease and has many effects that none of us will ever know. Maybe he should come alone to visit his mother to spend some quality time with her. She will die eventually from this disease. There is NO cure. All treatments are just for buying time. You just have to decide if the treatments out weigh the benefits.
I go to see my grandchildren whenever I am able to. That way I can leave when I have had enough. She can't go, as she is in the last stages. My priority now is my mother. I have to side with her and her feelings as I am sure your husbands father is doing as well. Your husband needs to accept the course this disease will take and enjoy the time he has left with his mother. You can explain to your children as much as you can for their age how their grandmother is sick and when they are at her house to be on their best behaviour and keep any family visits short.