Academia is based on meritocracy and though there is snobbishness towards former polytechnics, they will respect you if you do good research. I think the trick is do good research, go to conferences, publish, make good connections, get more publications, post-doc at a better uni, do more good research, succeed. You do have to work harder than students at more well funded universities but you should focus on what you can do (good research!) instead of missed opportunities.