I submitted a paper to one of the most prestigious journals in my field. Surprisingly, it has been given 'major revisions'. The reviewer comments are not too difficult to address, but an associate editor is not convinced that the paper adds enough to the literature. The editor said that I could (1) scale back the paper and aim to publish in their 'short paper' section or (2) take the riskier option of trying to convince the AE of the novelty and revise as a full paper.

The AE's criticism is very reasonable and I was planning to take this research in the direction they suggested anyway. I was thinking of hunkering down and extending the work, then revising as a full paper. This will take a lot of effort and there's no guarantee I will be able to do it. I've gotten some advice to just aim for the short paper option, since it's still a really good publication and saves me time.

I just wanted to get a second opinion on this. I'm scared of spending a huge amount of time addressing the AE's comments, only for them to not be satisfied and reject anyway! Has anyone been in a similar situation?

If you don't like the revisions requested you can always submit it somewhere else.

Yes that's true. I think I'd be tempted to go for the short paper though, if your priority is to get a paper out of this fairly quickly and in a great journal.

If you see it as a larger paper then put the work in, but you'll have to accept (as you're already aware) that it might not go in that journal. Do you have back up journals and would you be happy for it to go in one of those (hopefully) if you put in the work and it isn't accepted in your first choice?

Thanks for the advice! There are other journals I can sent it to, although we really targeted this one since the work I'm building on was published there. I might give it a couple weeks to see if I make any progress but, if nothing opens up, maybe I'll just go with the short paper...

It's annoying because normally I would be happy to spend time trying to deal with these problems, but I'm aware of the need to get first author papers on my CV by the time I go on the job market...

Beware of merely submitting elsewhere without making the changes requested.
You might end up with the same reviewer again.
It happened with a paper I worked on in a collaboration.
The paper got rejected because of unsupportable claims the main author made about my part of the work
He ignored my suggested changes and submitted anyway.
Got it rejected from journal A after months of waiting because of my work as expected.
Rather than talk to me he submitted it unchanged to a second journal amd it landed of the desk of the same reviewer.
I wish I had kept the email response from that reviewer.
At this point, the first author dropped this crap on my desk and asked me to fix it.

I made the changes I had first suggested before we even submitted, apologised profusely to the reviewer and we got it accepted straight away.
This is a prime example of why I LOATHE working in teams where I am not in charge. Reviewers need to be handled with kid gloves and I seem to have developed a knack of cutting through their abuse and ticking their checklist off. That first author was just an arrogant idiot.