This is probably an odd problem. Basically during my PhD I have tried to enhance my statistics skills and I'm quite curious about new methods, so I don't see anything wrong with trying out new tests etc.

Anyway, I'm now at a point where I've done my stats, but its quite advanced (in relation to the knowledge of people in my field). I've already submitted an article for publication where the reviewers have said they don't really understand the stats - I've explained it in very simple language too - their comments were more like 'we don't see the need to do this, why don't you just do a t-test' kind of thing :-s (I have clearly explained there is a need to do the more 'advanced' stuff to control for error etc). My supervisor is also not good at stats and can't understand what I've done.

SO now I'm scared - maybe I should do the simpler stuff, in case he more complex stuff (which is better and more rigourous) gets pulled apart in the viva because the examiners don't get it?

Sneaks, all power to you and what you're doing is what research really should be about - investigating and exploring new ways etc.

However, i'll be completely honest, at this early stage I think it's all about playing the game and giving examiners what they want / expect. I'm not going to do anything that challenges me too much. That said, as long as you know what you are doing, it is correct and you can explain yourself clearly you'll be OK, I should think.(up)

No! Stick to your guns! So many people don't care about statistics and that really sucks. Just make sure you can explain what you did and why you did it - often I frame it in terms of why I wouldn't just use a t-test etc. I think as long as you can get across the advantages of your approach, your examiners will accept when they're out of their depth regarding the exact procedure and won't be unfair about it. Personally I think the key is to have a strong message in your discussion of results - ok, sometimes you have to explain an effect in terms of the nuances of the way you analysed the data, but other that that don't make understanding the discussion dependent on understanding the details of the analysis if that makes sense.

I'm a bit of a 'play the game too' if I'm honest, particularly as I don't want to stay in academia really. I think you do so possibly it's more important.

This could be a scenario where a presentation at the start of your viva would be be good - it would allow you inform the panel, in more layman's terms the strengths of your approach and the stats tests you used. You could have comparison slides - 'a t-test' produced this result whereas my approach gave this...' Just a thought

thanks both, its an odd one, because you can't actually test my hypotheses in any other way - so it would actually mean dumbing down my hypotheses to suit a simpler method, which I don't think its right if you can do it in a better way.

Problem is when I say 'complex' or 'advanced' it really isn't. I think I may show the results to a more stats-savvy person, my sup isn't the best person to check the work IMO.

The method that got picked up in the journal article is really common practice in other fields, just isn't in ours. I also suspect that the reviewer let their PhD student do the review, which means they probably haven't got a clue - the comment they gave suggests this anyway

I'm with Melsie - stick at it. If it's an improvement on existing techniques, then you'll be a front-runner in this new development. If it was me I would get a stats person to look over it just to be totally sure, and ask them to help me construct a rigorous defense for using this method. Then I would put together a strong argument why t-test etc is not adequate (not quite the same as showing why your method is better - this would focus on things they will understand).

I totally see the issue with not wanting to jeopardise your viva, and only you can decide if the risk is worth it, but if your examiners are halfway decent they should surely be open to a new approach. Like I said, if all goes well, it's your work that will be cited when others follow.

I'd stick with it. If you can explain why it is better than a t-test then this should answer any queries from non-stats people in your viva. Agree with the idea of seeing a statistician too so you can add weight to your arguement.

Or could you do the t-test too then expand with the new test. Sometimes people are suspicious of non conventional tests because they wonder if it's been done because the straight forward test didn't give the answer you wanted.

Oh and just out of curiousity - what test is it?

well I used the example of a t-test, but basically all my studies are using bootstrapping, which a lot of people just don't 'get' no matter how easy an explanation I provide. I'm also using mutiple mediation i.e. so regression with 3 mediators in using Preacher and Hayes method, rather than baron and kenny. Its really not that difficult, but is just 'new' so I think it may cause issues.

I got a book out the library on boot strapping but still haven't got round to reading more than the first few pages. Sounds an interesting approach but I want an easy life! so have stuck to what I know.

I'd definitely stick to your guns with it. So long as you can justify it in your viva I can't see it being a problem. Do you have an opportuinity to answer the reviewers questions on approach? or was it an out right rejection of the paper.

I did at point consider using Bayesian stats but think again this has to justify itself for publication as people understand conventional stats.

It was a rejection, but not for the stats, actually my sup thinks they sent it to the wrong reviewers and is replying to the editor saying it should be an R&R. So we'll see I suppose.

At the mo I'm more concerned with whether my thesis examiners will take issue with the stats. I've yet to speak to my sup, she's just sent me an email saying she doesn't understand the latest results, so will speak to her 1st - a lot of it is that I sent her the results only and she can't think beyond that i.e. she says "you haven't included anything about your sample" - yes that's going in the method, which I haven't sent yet :-s

Hi Sneaks, can you reference another article where they used the same stats?

yeah I have done, I've gone pretty much all out saying "other people have done this too in our field, its fine, and its far better than using the old method - see these people say it is too and they have been published in really good journals so naaaaaah" but I fear it still isn't enough

I feel even more emphatic about it now that I know you're using bootstrap :p(up)

thanks - see its really not difficult stuff, I think its party old professors not wanting to learn anything new :p

I've decided to stick to my guns, go and find some MORE papers using all the methods etc. and then be ready to defend it in the viva!