To much DMSO can. I remember when I first started and misheard the postdoc. I put all my cells into 100% DMSO. Seemed strange at the time, but I had only just started out so felt I should just go with it. Oddly enough all the cells died. Though 20% should be ok.
You could try 10%, but I would be surprised if that did anything.
Maybe you just dont have many cells/vial you are freezing. Do you count and put a set number in? Maybe its too low (esp if viability is very low as well).
I have to say though, if you are not seeing much in the way of cell debris it seems likely that you are somehow not putting any cells in in the first place. Cells die, but they usually take a while to disappear, and the cell debris is likely to be maintained by the freezing, so you should see lots of bits of cells (though i suppose this depends on how they are being killed). What steps are you doing between the pellet and the freezing?