It happened the same to me although I currently own no rolex watch.
I thought rolexes were overpriced (ok, they are ), overestimated.
Now, after buying sevreral reps, I reckon they have a unique understated beauty that lasts forever and almost perfect measures.
At the same time they attract too much attention and I wouldn't be comfortable wearing a rolex rep.
My next Rolex needs to be a gen....
As for your question, I'd say go ahead, sell part of your collection if you don't wear and enjoy your watches that often.
Now I'm curious, would you please list the watches you wear the most?