Yeah definitely! I think it is even encouraged that you go to as many conferences as you can anyway when doing research, even if you're not contributing.
It's a really good way to network, get advice, make new friends, and open new doors to job opportunities.
It's probably a good idea to give a paper now and again so that others can see what you do and know that you exist, but if not, you could always volunteer to help organise one. This way, you will be at the centre of a network which would be particularly useful if you've chosen to help in one that is more closely related to what you are researching.
Go for it!