There is no rule of thumb that can prove Brady is the greatest of all time.
The fatal flaw in the Patriots legacy is SpyGate and then later DeflateGate. I personally don't think they needed to cheat. They are a good team that plays in an embarrassing division. But, their cheating put a dent in the integrity of the game and was the start of the fans' distrust in Roger Goodell. Ironically, many fans trust Goodell less after SpyGate because the team he helped cover up their cheating...cheated again and now paints Goodell as the bad guy.
I'm not the type that goes out an accuses people of cheating without good proof. But, we simply don't know what else the Patriots are doing that is obscenely illegal. If anything ever comes up, even as low as paying off referees, paying off opposing team employees for information, etc....it really wouldn't surprise me. And there will still be fanboys who will try to excuse this behavior.
In the end, they will ALWAYS be remembered for SpyGate and DeflateGate along with their Super Bowl trophies. If that doesn't stain their legacy, I don't know what does.
YR