It does matter and it matters even more over muddy bottoms or over grass
An M0 will lay on the mud and a M2 will sink in
The fish bump the bol around and are skittish around heavier weights. The bigger the metal signature the more spooked they are when getting through the bol
The rule of thumb is to fish as small a weight as possible for the distance you are fishing
ie. If you can cast to your spot with a M0 then use an M0 and not a M1.5 and if you can reach the spot with a P4 then use a P4 and not a 120gr weight