There are (at least) four different definitions of winter:

• Astronomical seasons are what you describe, and these are obviously based on astronomical events, for which the equinoxes and solstices are the perfect fix points. That the seasons start at these days is purely conventional, and in some times (roman empire) and places (Celtic calendar) people used these days as the center point of the season instead (also known as "Solar seasons"). Why those don't match up with the actual temperature has already been explained in countless other replies.
• Meteorological seasons use a simple, month based approach, where winter is just December, January and February. That makes it easier for statistical usage but obviously is also just a man-made convention.
• The energy sector defines winter as the time of extended energy needs due to heating. Where I live, that's defined as 1. of October - 31. of March, but for obvious reasons that's highly dependent on where you are.
• Phenological seasons are a bit more what you're looking for: They are based on biological events in indicator plants and a lot more complex than just being four fixed periods and only ever the same for small regions. Where I live, there are ten phenological seasons, winter starts when the English Oaks drop their leafs and ends with the blooming of the Common Hazle.

In other cultures there might be vastly different seasons. In many tropical countries you'll only have the dry season and the monsun season.

To answer the original question, it's because the earth is very large and retains the heat from warmer times preceding the solstice in the water bodies and the ground, like a buffer, so there is a lag time for things to lose that heat and get to the coldest air temps. As an analogy, if you heat up some rocks in the oven, then turn off the oven, the rocks will still stay hot for a long time and gradually cool off.

Several cultures do use it to mark the middle of winter. Celts and Chinese come to mind. They are communiting a different set of information when they do it.

Here's a ~3 minute video explaining why. TL;DW it takes time for the changing amount of daylight to affect temperature

Ever notice during the summer the hottest part of the day is usually around 3 or 4 in the afternoon instead of at noon? Same thing, just on the order of months instead of a day.

Take an ice cube out of the freezer. Drop it in a glass of water. Take the temperature of the water immediately after you drop the ice in. Take it again after 5 minutes. Which measurement is going to be colder?

Drop a red-hot nickel ball into a glass of water. Measure the temperature of the water immediately, and then again after 5 minutes. Which measurement is going to be hotter?

Basically, the surface of the earth holds heat, which causes the atmospheric temperature to lag behind the heat input from solar insolation. The cold surface is still warming long after the summer solstice; the hot surface is still cooling long after the winter solstice. The atmospheric temperature extremes are closer to the equinoxes than the solstices.

It's a social construct but it's based on the equinoxes/solstices being evenly spaced, somewhat objective, and easier than any alternatives.

Seasons are bullshit anyway

There are three popular reckonings for the 4 season model. The meteorological seasons, astronomical seasons, and solar seasons. The solar season places the solstices and equinoxes in the middle of each season. The meteorological seasons are based on temperature but these days are aligned with the months with summer/winter (depending on hemisphere) starting at the start of December and ending on the last day of February. The model the US uses is the astronomical model of seasons and makes the least sense to me, it places the start of each season on the solstices and equinoxes which means that midsummer is actually the start of summer there.

I once again come to inform that "the start of winter" changes depending on where you are.

So in you case that is just how it lines up, but where I live the winter starts October 14th.

I always thought it was because in mid northern hemisphere latitudes where most people live, the coldest day of the year (i.e. "the middle of winter") tends to be in early February, not the winter solstice.

Not sure if it is technically correct, but I think of it like the momentum of Earth's atmosphere. It takes a lot of momentum to alter the system. The days getting a little longer by a few minutes depending on latitude makes little difference. We are still dropping in average temps the further towards the poles.

Riding a bicycle everywhere for years in Southern California, this was something I would think about a lot on the commutes in the dark of winter, and the wind and rain patterns. Like here, I know if it will rain based on the wind direction alone. I only check the weather when I'm too lazy to go outside or something odd is happening. After the solstice there seems to be more turbulence that is added to the system. There is more of a back and forth between on an off shore flow patterns. It is our rainy season here, and we have nights get around 10°F cooler. This is when I break out my 40°F layers. On a bike, everything is still skin tight. I can shed some layers but don't like to stop to do that. I have specialty gear for every 10°F of temperature drop. If I wear 40-50°F gear in 50-60°F temps, I will sweat like crazy and then freeze from being wet. I wear my coldest gear a lot less than I did 15 years ago, but I still haven't needed it this year. I will in the months to come.

Most people are not connected to the weather and outside world very directly like this in the West. In my experience, the solstice marks the low point where I have around 6 weeks until things start getting better.

It’s a convention. And not one that everyone follows. In Australia we use meteorological seasons, which start on the first of the month. So today is precisely one third of the way through summer.

Because of what you said, I do think that the way we do it is better. It puts the solstice (the day literally called "midsummer") closer to the middle of the season, and also makes more sense because it puts seasons aligned with the human calendar.

Ultimately, the 4 seasons are a human invention and we can choose where we put them.

Edit: meteorological seasons start on the first of the month. The name for when you start on the solstice/equinox is astronomical seasons.

Solstice sure as hell isn't the first day of winter of all of us, we got first snow on October or November.

It's a modern convention. In the Celtic calendar solstice marked the middle of winter and older Christmas carols mark Yule as midwinter. https://m.youtube.com/watch?v=1_R3GPkIDeM