an unpopular comment posted on downvoting-enabled Site A will show higher when viewed on downvoting-disabled Site B

More or less yes, depending on how you have your sorting set and if the comment/post has any positive interactions. At worst it just shows as something with a single upvote and is sorted based on time or however it fits inline with other content with similar vote numbers. Since it's removing the outliers you see a better overall picture of what's being said IMO. It's up to you to decide if that's positive, negative, or even something you care about.

Or only if it was actually posted on Site B?

It affects how people on instances with downvotes disabled see it no matter where it was posted from. Basically the downvotes simply do not exist to instances that have them disabled.