The way I've seen it, it appears to primarily be used by the various British and former British colonies, including the US. For these groups, anyone from outside the colonies living in "our" territory is an immigrant (who is certainly a lower class!). However, if we choose to reside in another country, we are not immigrants, we are "expats".

Not everyone uses this term, but those that do frequently congregate in English speaking enclaves and make no attempt to integrate into their new home. They often see the locals as a sort of servant class, particularly because they probably came with enough money or income to make them wealthy by local standards.

As you might imagine, people with this attitude are probably not very popular with locals.