This should be handled by (auto)mod tools, allowing good faith users to help filter non-fitting or low quality posts, and preventing bad actors from downvote fireworks.

There are possibilities to limit available downvotes either by throttling amount per time/community/instance/user, by demanding to have a certain “reputation” in the community/instance or by making downvote cost your own “reputation”.

None of the above is perfect, but IMO outright removing a right to disagree is against community interests.

Currently I can either leave a “harassment” comment: “wtf this is doing in this community”, make complain to a mod (if there is any active), or ignore the post. Neither help community grow stronger, I think.