I've used the Windows task manager to better understand how much memory was used by the different views of Elvenar, and then compare with another similar game which also has a city view and a world map. I'm not going to give you the numbers, it's boring, but only information about the variations of these numbers.
First, hardware acceleration is not to be spoken about, it only saves a small amount of memory, but doesn't change the main thing.
The main thing is that the world map is a memory hog. The world map is graphically much simpler than the city view, so I expected it to use less memory. This is true with another game (-20%), but in Elvenar it uses more (+30%) which I can't understand since there are less image components, no animation, and the pattern is repetitive, so how come ?
Besides that, I also noticed that memory consomption increases over time, either by switching between views, or just by waiting, but I must say this also happens with the other game, so I wouldn't concentrate on that first, even if it's not normal.
In my opinion, the world-map-not-loading problem may be related to an abnormal memory usage that has to be understood first.
Please understand that I'm not a specialist, I just think what I observe should make sense.