That's the thing. Being just as fair doesn't necessarily imply it's equally travelled. Even being worn the same doesn't necessarily mean equally traveled, although it strongly implies it. I think the final line is so certain that it overrides the earlier lines and implies to the unwary reader that these similar paths actually were differently travelled.

I don't expect self contradiction in a story / poem. So that certainty of there being a difference overrides all.

It's only after reading the author's intentions that I know for sure that the contradiction was intended and that was actually the point of the poem.

As I said before, this makes me like the poem even more now.