While I'm not sure if there's a specific rule requiring it, generally speaking people expect that an upgrade (something needing extra points not included in the basic description of a model) should be modeled appropriately onto the dude/lady/thing.
But Chosen
CSMs come standard with bolt pistol,
CCW and bolter. So you can model them however you'd like, with a
CCW/pistol combo or boltgun. But being Chosen, since they're 'fancier', you could stick a bolter slung onto their shoulder whilst posing in a cool manner with the other standard weapons. Or if you have one with a melta gun or something like that, you could stick a chainsword on the belt for a complete effect. Aesthetically, it does improve the air of awesomeness on most models, and therefore performance
If you put on a power weapon, modeling that as an axe, sword, or stave, or lance will make a big difference...that's the only instance I can think of where the actual model piece is needed, but again, those are upgrades.