I'm studying for my PhD at Durham. As far as colleges go I'm a member of Ustinov (the default option for postgrads), but I virtually never go there or have anything to do with it, and I am also a pastoral tutor and member of the SCR at St Mary's.
Your decision will probably be influenced by whether you want to live in (I didn't), and to what extent you want to be involved in college life (I didn't really care either way). If you're coming from far away and won't be able to look for accommodation, it's probably better to go for Ustinov because they can guarantee you a place in halls with lots of other postgrads, and it's not a massive hassle to move out if you don't enjoy it.
What are you going to be studying?