One thought is to try user blocking Lemmy.ml. Unlike an instance block a user one is not sufficient bc you can still view the comments, and I'm pretty sure that they can still downvote you (occasionally they have been known to coordinate mass downvoting campaigns on Matrix too, people say), so all it really does is block communities from those instances from showing in your All feed, and if someone replies to you then you won't receive a notification to that effect. Otherwise it's back to blocking users one by one...

I wish there was a major, known instance somewhere that defederated from Lemmy.ml. Even Kbin.Social does not though.

It will indeed be interesting to see how Sublinks will improve things.:-)