I didn't think you could punch the guy. But I'm not certain. You may not punch the ball.
Here's the context of the rules: let me know if there are any problems with this.
Use Of Hands, Arms and Body
Rules and exceptions
1. No player on offense may assist a runner except by blocking for him. There shall be no interlocking interference.
2. A runner may ward off opponents with his hands and arms but no other player on offense may use hands or arms to obstruct an opponent by grasping with hands, pushing, or encircling any part of his body during a block. Hands (open or closed) can be thrust forward to initially contact an opponent on or outside the opponent's frame, but the blocker must work to bring his hands on or inside the frame.
Note: Pass blocking: Hand(s) thrust forward that slip outside the body of the defender will be legal if blocker worked to bring them back inside. Hand(s) or arm(s) that encircle a defender -
i.e., hook an opponent-are to be considered illegal and officials are to call a foul for holding.
Blocker cannot use his hands or arms to push from behind, hang onto, or encircle an opponent in a manner that restricts his movement as the play develops.
3. Hands cannot be thrust forward above the frame to contact an opponent on the neck, face or head.
Note: The frame is defined as the part of the opponent's body below the neck that is presented to the blocker.
4. A defensive player may not tackle or hold an opponent other than a runner. Otherwise, he may use his hands, arms, or body only:
(a) To defend or protect himself against an obstructing opponent.
Exception:
An eligible receiver is considered to be an obstructing opponent ONLY to a point five yards beyond the line of scrimmage unless the player who receives the snap clearly demonstrates no further intention to pass the ball. Within this five-yard zone, a defensive player may make contact with an eligible receiver that may be maintained as long as it is continuous and unbroken up until a point when the receiver is beyond the defender. The defensive player cannot use his hands or arms to push from behind, hang onto, or encircle an eligible receiver in a manner that restricts movement as the play develops. Beyond this five-yard limitation, a defender may use his hands or arms ONLY to defend or protect himself against impending contact caused by a receiver. In such reaction, the defender may not contact a receiver who attempts to take a path to evade him.
(b) To push or pull opponent out of the way on line of scrimmage.
(c) In actual attempt to get at or tackle runner.
(d) To push or pull opponent out of the way in a legal attempt to recover a loose ball.
(e) During a legal block on an opponent who is not an eligible pass receiver.
(f) When legally blocking an eligible pass receiver above the waist.
Exception:
Eligible receivers lined up within two yards of the tackle, whether on or immediately behind the line, may be blocked below the waist at or behind the line of scrimmage. NO eligible receiver may be blocked below the waist after he goes beyond the line. (Illegal cut)
Note: Once the quarterback hands off or pitches the ball to a back, or if the quarterback leaves the pocket area, the restrictions (illegal chuck, illegal cut) on the defensive team relative to the offensive receivers will end, provided the ball is not in the air.
5. A defensive player may not contact an opponent above the shoulders with the palm of his hand except to ward him off on the line. This exception is permitted only if it is not a repeated act against the same opponent during any one contact. In all other cases the palms may be used on head, neck, or face only to ward off or push an opponent in legal attempt to get at the ball.
6. Any offensive player who pretends to possess the ball or to whom a teammate pretends to give the ball may be tackled provided he is crossing his scrimmage line between the ends of a normal tight offensive line.
7. An offensive player who lines up more than two yards outside his own tackle or a player who, at the snap, is in a backfield position and subsequently takes a position more than two yards outside a tackle may not clip an opponent anywhere nor may he contact an opponent below the waist if the blocker is moving toward the ball and if contact is made within an area five yards on either side of the line.
8. A player of either team may block at any time provided it is not pass interference, fair catch interference, or unnecessary roughness.
9. A player may not bat or punch:
(a) A loose ball (in field of play) toward his opponent's goal line or in any direction in either end zone.
(b) A ball in player possession.
Note: If there is any question as to whether a defender is stripping or batting a ball in player possession, the official(s) will rule the action as a legal act (stripping the ball).
Exception : A forward or backward pass may be batted, tipped, or deflected in any direction at any time by either the offense or the defense.
Note: A pass in flight that is controlled or caught may only be thrown backward; if it is thrown forward it is considered an illegal bat.
10. No player may deliberately kick any ball except as a punt, dropkick, or placekick.