I'm assuming you are in the US; most people on here are from the UK.
I know a couple of people who had 20 hour GTA positions - they still had to work 40 hours a week on their PhD on top of this, so obviously with a child it's going to be very difficult.
Having said that, you wouldn't be the first person to do this, so if you want to do it, I'm sure you can.