The designers chose this setting and lore, and could have chosen otherwise for the sake of game experience.

Additionally, there's no reason for the fast travel to have to be distinct, separate from gameplay, as loading screens.

Elite Dangerous keeps you in your cockpit, replaces the outside view with an animation while it loads the system you're jumping to. When landing on a planet, there are various "entering the atmosphere" effects on suitable planets to mask swapping from space to the landable planet.

For ED, in-system FTL is time consuming and you can shave off around 25% of the travel time by doing it manually (risking overshooting and having to loop back around), or you can have the ship's computer do it. ED is multiplayer and you can be yanked from this "supercruise" by players and NPC pirates, so it works mechanically to make the player waste time with it. In Starfield they could show you the ETA and give you the option to skip it or to wander around your ship during it while the ship does its thing.

If you're in a menu on your ship when FTL would end with autopilot, stop the clock before leaving FTL, pop up a message in the corner saying the ship is ready to drop from FTL, and let the player exit it manually from the cockpit so you can't get ambushed while you're on the other side of your ship.

No changes to setting or lore needed, except that there's a basic autopilot now.

As far as programming that goes, the engine already uses loading during gameplay when you're on the overworld, and they have done that since Oblivion. Overworld is set up in chunks, they keep a certain number in each direction around you loaded, and load/unload while you move around.

I won't say it would be easy to expand that background loading functionality, but I will say that they've had many many years to attempt it.