Hi Sherlock,
good question. I would say that the university's rank does 'matter', but no to a great extent. As you said. that your subject and supervisor will offset any low rankings, but also, a uni which might have a relatively low place on the league table, may excel in a certain field, or have good a good research background (look at the research assessment exercise RAE scores).
I am doing my PhD at a uni overall ranked sixty something, but has excellent RAE scores, my supervisor is a big name in the field nd very experienced and I just 'like' it. I reckon its more important to go with an educated gut feeing- if you have a better good 'feeling' about the uni and supervisor etc then perhaps don't focus on ranking too much.