have to do a lot more of the same thing in every compositor than window managers ever had to

Yes, but is that not entirely expected? As far as I understand, compositors are complete implementations of Wayland's display server specification, whereas window managers are just a helper program that, well, manages windows, while Xorg does the heavy lifting required to fully implement the X Window System protocol. So the only real difference that I see is that, in the X world, the "common parts" are managed by a separate process (Xorg), whereas in the Wayland world, they are managed by a separate library (wlroots). So a hobbyist developer trying to make a window manager in some obscure language would need to figure out how to communicate with Xorg in that language, whereas a developer trying to make a compositor in some obscure language would need to write wlroots bindings for that language. Maybe I am just ignorant, but those seem like comparable efforts to me.

And lastly, in the X world, the only (widespread) implementation of the X Window System protocol is Xorg, but, in the Wayland world, there are compositors that use wlroots, and those that don't. So wouldn't that alone indicate more fragmentation / diversity? Sure, there are more X window managers than Wayland compositors out there, but X11 has also existed for longer. In short, I don't see how the Wayland system is more adverse to diversity of implementations than X