The simple fact is that we've beaten three teams that don't have functioning offenses. The Giants and Cardinals wins don't look so impressive in retrospect, and the 49ers are a terrible team and had a QB starting his first game.
As for our losses, I'm willing to write off the Denver game as a fluke. We lost close ones to the Rams (who are very much for real) and the Packers (who were very much for real at the time) missing big pieces from our defense.
The offense is fine. As I predicted before the season, there was much hand-wringing in here over the offense after opening up against VERY tough defenses. But it's also true that the offense is looking more cohesive every week. The 49ers weren't much of a challenge, but we did exactly what we're supposed to do against bad teams: we destroyed them with 500 yards and 40 points.
I have no idea what to expect from the defense and I don't think any of us do. We have yet to face a competent offense with Lee, Hitchens and Irving on the field (and Jaylon limited to a small number of snaps). Also, our rookies have a few games under their belt now. The next two weeks should be very interesting tests. (And maybe the Falcons the week after that, if they've fired Sarkisian be then).
Losing close games to good teams (Rams, Packers) isn't by itself a big deal: good teams lose a few games, and the games they lose are close games to tough opponents. The problem is that we lost two of those kinds of games early and those losses are in the books. Our margin for error is vastly reduced going forward. If we'd won one of those games, it would be easier later in the season to shrug off, say, a close loss to the Chiefs as a "these things happen" game. We can't afford those anymore: we've used them up already.