It is. There are many stats that must be evaluated to truly assess a D, or anything else for that matter. As a standalone stat, yards come the closest, which is why the NFL uses yards. The majority of fans are unable to comprehend complicated stats to truly assess. The fans on this board are mostly exceptions to that rule, as the fans that reside on here are way more into football than your everday run of the mill football fan.
You can use points allowed to assess how your team is doing, no doubt. However, if you are trying to assess just your D, yards is more accurate than points.