The songs aren't that great (for the Beatles, anyway)
There are at least three songs on Sgt. Pepper that are better than anything on Abbey Road. Come Together is overrated. The George songs are great but there are only two of them, and most of the pieces of the medley wouldn't stand up to most Beatles songs on their own. If you compare an album like Revolver song by song to Abbey Road, it's not even close. And I'm not sensing the cohesion of Rubber Soul or the adventurousness of Pepper or the White Album on Abbey Road either. Rock critics can be jerks though, and again I must stress I am not one at all and absolutely appreciate the album as a whole. But we'll continue
Lennon mailed in his performance
Abbey Road and Let it Be are probably John's weakest albums with the Beatles, and this happened after he'd shown his dominance once again on the White Album. Sure, you could debate that he was still the leader of the band at this point, writing and singing as many songs as anyone. But I tend to prefer Beatles albums where John is on fire and writing most of the songs (Rubber Soul, most early albums etc). Or on an album like Pepper where Paul may have written more songs, John made up for it by writing a few of the best songs we've ever heard.
His bad car accident right before the start of the sessions could probably explain his disinterest, so there's a valid excuse at least. But it still doesn't make Abbey Road a top four Beatles album.
McCartney was being a control freak
Paul was probably the sole reason the last couple of Beatles albums were even made. And we can thank him for that. He may be the most ambitious rock musician alive today. But as much as I love the medley on side two, Maxwell's Silver Hammer is one of the very worst Beatles songs. I'm not an Oh Darling fan either. The fact that those were chosen over All Things Must Pass and other George tunes was a bit arrogant. Let's just say it didn't help hold the unit together. It probably was time for them to go their separate ways, at least for a while. The sad thing is, it appears they were nearly ready to pick up where they'd left off when John was killed.
Harrison was the only Beatle who was "on"
And let's not forget many of George's best songs were being passed on by this point. His experiments with an awful-sounding Moog synth at the time didn't help things much either. Sure, "Here Comes the Sun" and "Something" are two of the greatest songs ever. But George having the best songs on a Beatles album is just plain wrong to me. Imagine if he had the two best songs on Pepper. That would have been impossible for anyone. At the same time, his guitar playing on Abbey Road was maybe the highlight of the album.
You like it for its production?
The term audiophile is subjective. Many people point out the fact that the Beatles now had more tracks to record on at Abbey Road Studios (which is an ultimate destination for people like myself, but was becoming a prison for the Beatles by 1969), and George Martin's finished product sounds sonically better than previous Beatles albums. Well on the other hand, maybe slightly because of this, I think Abbey Road barely even sounds like a Beatles album. Sure, it was a big step up from Let it Be, despite a few classic songs being on Let it Be. But I see it almost as a reunion album - It's just the reunion happened very quickly before the guys in the band all started to completely despise each other... not a great atmosphere to record an album in. I can't help but feel down listening to it sometimes.
So I guess these clouds surrounding the Abbey Road album make it the Beatles' ultimate triumph to some people. Personally, while it's all just opinion, I think if someone likes it better than Rubber Soul, Revolver, Sgt. Pepper or White Album they've overlooked something. As weird as I am, I like Beatles For Sale and some days even Help better than Abbey Road. And let's not forget that while the Beatles first few albums are not among their best, they were easily the best anyone was putting out at the time. While Abbey Road had some stiff competition. But what do I know? I did name my cat Abbey after all, although I do frequently point out that she's named after the most overrated Beatles album. Please let me know how wrong I am!