I've worked between 15 and 20 hours a week in my part-time job for most of my PhD, and I've done teaching and exam marking and stuff on top of that (teaching and preparation took maybe an afternoon a week averaged over the year). I've given up work over the summer now while I finish writing up, and I'm hoping to be finished by Christmas (so that'll be three years and a term). It's do-able, but don't expect to have a spectacular social life! I don't think my thesis has suffered, but there's extra things in the dept which I haven't been able to do that would have looked good on my cv, but I just don't have time for.
I was also lucky that I had a relatively stress-free, often quiet, and well-paid job, so I sometimes got £8 an hour for a ten hour shift, of which six hours were spent reading. Sadly, my job has changed now, which is one reason I've quit for a few months!