Generally yes.

GIF's ancient LZW compression is remarkably ill-suited for modern CPUs, and more expensive than modern algorithms. Combined with significantly larger file sizes, it costs much more to decode, on top of increased costs of transfer and caching.

GIF might have an edge if the animation is very small (<16px, few frames).

It also gets messy if you need to play hundreds of animations. GIF will be terribly inefficient, but also browsers aren't designed to have hundreds of video elements, so both will eat memory in their own way, and it will vary which is worse.

WebMs encoded with VP9 or even better AV1 are better than GIFs. Decode speed is negligible, especially considering how much more quickly it will load

IMO webm files are ideal as they're extremely small for the quality of video output. Downside is, they can take a while longer to encode compared to mp4s if your hardware can't accelerate it

What kind of animated site elements?

A lot of "gifs" are actually served up as mp4s actually

Follow-up question: How about transparent animated graphics?