For me... I could have picked up some nice gens for what I've spent in this hobby, but to me, there's not a lot of fun in doing that. I'd rather spend the time to research a build, understand the history of such a piece, hunt down the parts and see what I can put together with my own hands and tools. I have an appreciation for what I have on my wrist, and I know what went into it.
In terms of cost... I think that's all relative, right? I have more than I'd like to admit into my Patek builds, but the amount is a small fraction of what the gens would run me, and that's if I could find examples as clean as the ones I've put together. For me, maybe that's the other facet; building something that's not the most accessible or available in gen form.
And sometimes... It's about pushing the limits; to see how close or accurate a build can get.
In the end, only you can justify what makes sense, or what doesn't. If it makes you smile every time you check the wrist... If you look back on a build with no regrets... I think it's nothing to worry about Life's short... We should all do what we enjoy doing!