Adding this kind of details, I see nothing bad with it. If you have the bits and skills to do it so it looks "natural", go for it.
In this particular case, I would add the gun for sure to show people he has a boltgun. The scabbard is kinda pointless gamewise, but will make a nice detail.
I personally try to customize my characters to a similar point, although with heroic scale weapons and guardsmen it is not that easy, as everything is ridiculously large. Adding pistol holsters or slings to rifles is the least thing you can do if they are not actually holding the gun in combat position. (although I suffer from a dire shortage of empty laspistol holsters when it comes to my sergeants)
Is it an OCD? If yes, so be it and welcome to the club. You'll be known as "the guy who's models have it all"