I understand what you're saying.
I have never ever owned a Rolex, nor have a changed a bezel insert.
This is what I'd do:
If the inserts look almost the same (except for the pearl), I'd switch the whole insert.
If the bezel that arrived with the watch is better looking, then remove the "pearl with it's little silver bezel", and switch to your better looking "pearl with it's little silver bezel".
To remove "pearl with it's little silver bezel", I'd use some plastic or wooden material (a spatula perhaps) not to scratch the insert. And then press gentle from the side of the "pearl with it's little silver bezel". Make sure you don't lose it, when popping off!
This is just a suggestion, and what I'd do.