Any thoughts on the following would be greatly appreciated. Conducting mixed methods research in social science. Compiled survey and adopted existing measures based on lit review at that time. The survey results were not particularly astounding and my supervisor and I felt my model wasn't really working. So changed the model and now feel like I have a model that can be used and defended! However, it meant going back to my survey results and omitting a couple of the measures used and adapting others e.g. reducing a 20 item measure to two measures of 2 and 3 items each. The results are sound but I'm really worried how to defend this in a viva. Am rewriting my LR now based on the new model and am scared that the whole thing will be seen as one major 'back fit'... Has anyone any experience of this? My supervisor says basically I'll have to come up with a way of defending this but I'm not sure how! Looking through Creswell's triangulation design at the moment trying to get inspiration but it's not really happening. I need to submit by Jan so this is not good!

======= Date Modified 05 Jul 2012 14:13:21 =======
I'm not sure if I've totally understood what you're trying to say - I am assuming you're worried as you've tinkered with your questionnaire AFTER participants have taken part to skew your results in the direction you wanted? If that's wrong, then please say so!

If you're really that worried, you do have time to adminster the survey again. As far as I understand it when it comes to using surveys, as long as your survey was reliably and validly designed, and you can prove this, then you have very little to worry about.

Are you planning to discuss the original model and why you think your new one works better? If so, I can't really see a problem with that - as long as you carefully discuss why you made every decision, and can point to some stats that support what you've done then it shouldn't be an issue, I'd have thought. (Of course, it really does depend on the field you're in, so don't take my word for it alone!)

But for example, if you've done a principal components analysis (or something similar) on your original 20 item measure and the results of that supported the new measures you've designed from that, or something along those lines, then that would look strong to me. Or even justification within the literature, if it's based on other people's empirical work using the same (or similar) measures and you can show that your decisions are based on something rigorous. I think you'd only really be in trouble if you'd just gone with an 'I reckon' kind of approach and ignored the stats, but it sounds like your results are stronger than that.

The beauty of mixed methods is that there is other 22 different definitions of it and so much potential for designing your own particular approach. All of the great players in mixed methods strongly caution that researchers should not feel pinned down by existing models, encouraging the development of new designs and approaches. As someone who is probably well read in the area, I don't think I need to remind you of the fundamental principle of mixed methods or its philosophical bed-partners. Therefore, do not feel constrained by the existing typologies because that is all they are! Ask Tashakkori and Teddlie...
I have difficulty understanding why you say the model was not working... What were you trying to do? Triangulation? Development? Expansion? It's usually not a case of your model not working, more it not actually fitting what you have done. Sound to me more like issues with the methods of data collection and analysis that you have now changed around (whether in terms of sequence or priority, etc) and now found that it doesn't actually fit your original model.
It's definitely a bit of a bugger for you, but not the be all and end all. As I said, you don't have to be tied to someone's model (they've only been defined to help make things easier for us novice researchers). Ultimately, you have to be able to show that it is indeed mixed methods - you can show this through both how you've combined the qualitative and quantitative data and through your worldview/personal research paradigm. To be clearer on this, take a look at how an education researcher called Anthony Onweugbuzie conducts his mixed methods research. Sorry if I have been unclear.

Thanks so much for all your help - appreciate you taking the time to post and have gotten some great advice so fingers crossed!