You're actually not correct.

65%+ of residential properties are owned by the family that lives in them. Another good chunk is dedicated rental apartment buildings, less than 15% of residential properties are owned by multi-owners and a good chunk of those are just people who have two properties.

This isn't necessarily true in the US, but it's accurate for Canada.

The vast majority of people who are making money off property in Canada are essentially just regular home owners (including me) and they're padding their retirement with essentially a tax upon the youth. I've made over $1 million in the last 14 years in equity gains, far more than my wife and I have earned after taxes working full time, and we only ever owned the home we lived in (though my wife and I have upgraded twice as we had more kids) and THAT is what's causing most of the problems.

A lot of the food, wages, and other problems wouldn't really be an issue if cost of housing had not gone up. Hence why most people who outright own their homes aren't really worried.