If you are leaving your first PhD for a second one, yes, it will definitely look very bad to your potential PhD interviewers. They will highly likely get turned off.
You could say that you were doing some research, but they will likely want to talk to your supervisors to verify. Do you have a friendly postdoc who could say some nice things about you? Did you do anything else during that time? Like volunteering, casual work, etc?