Your computer is still using the power and it can damage it

Thats not how that works.

Sure it could cause some data loss, or corruption, or other software issues if the computer was in the middle of something like applying updates, but it should not cause hardware damage. All this applies to holding the powerbutton as well by the way.

or (really unlikely, on a shit connector) you get shocked.

Then you have bigger problems. You shouldn't be using a connector that's a shock or fire hazard in the first place.

But seriously about the only bad thing about it is perhaps some wear and tear on the connector that's not designed to be plugged in and pulled out every day. Alternatively you could also "pull the power" by pressing a button on your powerstrip (if you have that), or by flipping the powersupply button at the back of the computer to off. It all does the same thing: it cuts the power to the computer instantly.

It’s also more convenient to hold the button instead of having to unplug and re-plug.

Yes, but holding the button is not instant and it relies on a software function in the bios which can be buggy. Usually holding the button doesn't even complete poweroff the system but puts it in a special "standby" power state where the motherboard still keeps providing power to some components. There are some issues that can only be resolved by a complete poweroff.