i think that its to be expected!!!! I went to houston with my supervisor and we skipped a day and went to NASA, anoter conference, my other supervisor told me to go out shopping etc.
I think that if you plan your time well you could go sight-seeing etc. I am going to one next week and that conference has organised sight seeing as part of the schedule.
You will not be interested in everysijngle talk etc.
Go to the dinners etc as thats when you'll get the most networking done. Theyre meant to be fun - so enjoy it!