I really dislike that German system, but for those that want an explanation:

Traditional European music theory evolved towards using sets of seven notes out of twelve in an octave. We eventually labelled those notes A through to G. Originally A was the lowest note available in common notation and we built our instruments accordingly (see the lowest and highest note on most pianos even today), but we then take a particular liking to the scale that starts on C using this system.

Even though this worked really well most of the time, in each seven note scale there was one standard combination that was pretty harsh (the diminished chord, such as the B chord in C major). To get around this, people just kind of accepted that B could be in two different places - the usual position if that sounded better, the flattened one (one twelfth of the octave lower) if that worked better. The system of sharps and flats wasn't standard yet - nor was the modern staff system at all, for that matter - and it was only really this note that it mattered for most of the time, so the solution was to write the letter B in two different ways depending on which one you meant. There was a round B and a square B.

And then Germany gets really good at making printing presses. This is very useful for spreading copies of musical notation, but it does present a problem: your press probably doesn't have two ways to write the letter B. So what do you do instead? Use another letter for one of them. H is the eighth letter, and it even looks kinda like the square B anyway, so that becomes the standard practice.

One fun quirk of this is that it permitted Johann Sebastian Bach to write his last name in musical form, which he went on to do in a whole bunch of his compositions