My conference was on a very niche topic, and as GrumpyMule says it makes it easier not having to run anything by anyone. I was free to organize it how I wanted, which I thoroughly enjoyed. I was also off-campus, but making arrangements by email easy enough. Having some admin support really handy though, but department will probably have someone to help.
Pitfalls I remember:
Not good enough signage for delegates to find the room.
Room too small in the end!
Water I ordered didn't arrive, should have had catering number at hand.