On the contrary, there are many who believe that larger sized watches offer several advantages. "Large watches allow subtle patterns or decorations to be seen and appreciated more easily. The dials, markers, and hands are much more readable. It is much easier to deliver accuracy and reliability in a large watch than in a small watch and shock absorbing, magnetic shielding, water-sealing, finishing, and assembling also become easier with thicker movements and larger cases."
I don't buy into the idea that the majority of people wear oversized watches to stave off insecurity. There are many small watches 38mm or less that in the $20,000+ range. In the end, like you said, it is all a matter of preference. If you like the Omega, go for it. (despite what women will think of you)