If you're going to have a complicated story you must work to a map; otherwise you'll never make a map of it afterwards.
I wisely started with a map.
The world is not in your books and maps, it's out there.