I'd say go with the supervisor too, over and above academic reputation. I was in a similar position, nearly half-way through my part-time PhD, but my supervisor was moving 500 miles away, and I wouldn't be able to go there very often to meet him. In the end I decided to stay registered with my current university, partly because I rely on extra support as a disabled student which is easier to arrange locally, partly to make things like library access easier. My supervisor stayed as my official supervisor, communicated long-distance with me, mainly by email, and another member of staff stepped in as a joint supervisor when I reached the nasty writing-up stage and needed more face-to-face contact and more regular meetings.