It works that way because as you said, it only appears at a certain angle between the sun and your view.

As you move, so does the rainbow. Because the place in the sky that matches that angle, also changes.

So it is happening everywhere the rain falls into direct sunlight. You only see it in one spot, because that's the one spot that matches the angle for you specifically.

You're almost there. It has to do with the angle of the sun and the water drops relating to the view point. Rainbows only show when the sun is behind you, and if you imagine a cone going out from the viewpoint outwards you get the possible paths of the rainbow (different radius different wavelength and therefore color)

A similar concept happens in certain reflective surfaces (metal pots and pans, car hoods and much more). You always see the tiny scratches in circles, but if you alter the angle in any way you keep seeing different scratch circles. This is because the circle you see in any given angle is the exactly the scratches that are turned just perfectly to reflect the light in the perfect way. It does not mean that the scratches you see at any given moment are the only ones. It means there are plenty, and only a few more visible at a time.

To me, playing around with the second concept (much easier to manipulate yourself and see) made me understand rainbows much better.

It does fill the whole sky. You only see the arc because our eyes can only see a limited range of colors.

Not an expert, but here's my understanding:

A beam of white light contains many wavelengths. If it hits something that bends the light it causes the different wavelengths to bend at different angles. The light ends up coming out as a rainbow with each wavelength being bent to a different degree. Not all of these colours might hit your eye.

If you have a whole bunch of prisms in the air, they all separate and scatter different colours of light in different directions. The red light from one prism might hit your eye but the other colours coming from that prism might not. The orange light from another prism might hit your eye but the other colours do not.

A rainbow is a pattern created by red light from some prisms, orange light from others, yellow from others, etc. You only see one colour from one prism, but together they form a pattern. If you move the rainbow moves too. Prisms that were sending red light into your eye are no longer doing so. Others that missed your eye are now hitting it. The pattern stays the same but it's being created by different prisms now.

You know more about rainbows than I do. I thought you were going to ask how rainbows are formed, and I was going to say magic.

This is a cool explanation:

https://youtu.be/MGyKQvSzdHE?si=-LV8NS0XRcJj961Q

This answer is purely fictional:

Imagine all the rain drops forming a film of glass. Now this glass is curved like a cylinder since rain is basically water flowing from a pipe in the sky. So you are looking at the sun through a curved glass hence you see the curved bow. Now your eyes are spherical in shape, hence the light reflected off the cylinder will appear in colours. Go outside and look at any cylinder, you'll see rainbow colours.

It’s explained pretty well in Optical Physics for Babies. Take a peek in the kids books section next time you’re at a book store. It also explains double rainbows, and why the second one has the colors in reverse order and is always more faint. This is a detail they got wrong in my favorite episode of Bluey: Rain.