The game gives you a lot of freedom in terms of gameplay and upgrades. You can finish the whole game without developing the city, but that's not advisable. Each building provides some benefits and makes things easier. Below, you can find a list of all in-game constructions along with a score awarded for their usefulness.
The main location in the kingdom. You can talk with the team members here, visit the armory or develop the city from here. Clicking on the globe allows choosing the destination of another expedition. The castle cannot be upgraded and can do everything it does from the very beginning.
Here lives a merchant who has lots of useful stuff: accessories, weapons, baits, etc. Access to this one is unlocked in chapter 1.
How useful? - 5/6 - The amount of items that Haksun has in his shop is pretty impressive. You definitely should maintain good relations with him and upgrade the store to get even better stuff.
This little inn is the best inn in the village. It's owned by an enigmatic man and his talkative daughter. You will unlock this building very soon - right at the beginning. During the week it's stormed by different characters - you can find most of the in-game characters in the inn.
How useful? - 3.5/6 - upgrading the inn lets you improve your relations with Lilka and Baz. This will mean increased income from expeditions and in bonuses for conversations that improve relationships.
A beautiful place, a showpiece of your town.
How useful? - 6/6 - You can upgrade this place only once, but the bonus yielded by this is significant - it allows you to take one more person for an expedition. The campaigns, especially later in the game, are quite challenging, so another pair of hands will be invaluable.
Here lives Miri, who will give you many sidequests, all of which will improve your relations with her. You can build the Study in chapter 2.
How useful? - 6/6 - Miri offers some of the best bonuses in the game for staying close with her. She can provide a bonus for finding good equipment after battles and unlocks highly useful passive combat abilities (bonus to damage, dodge and accuracy).
A rather peculiar and eccentric scientist may inhabit this building. You can buy here elixirs and other items that will aid your allies, weaken enemies, etc.
How useful? - 2/6 - Although you can't deny those items are useful, it's much better to invest in fishing or blacksmith's equipment.
Initially, the Pier has no real function, but after upgrading to level 2, it allows you to go fishing.
How useful? - 5.5/6 - Upgrade the Pier just once and you'll be able to start fishing. Fishing may be pretty time-consuming, but the items you can obtain are usually very good. Besides, fishing is one of the activities required to complete the kingdom quest.
There's no city without a good blacksmith. He get's there pretty soon, in the first chapter.
How useful? - 5/6 - Upgrading the blacksmith is a very good idea, as is keeping good relations with him. He can provide the best weapons and skills related to increasing damage.
An elfish girl will move in here in chapter 3. She sells accessories that increase the strength and resistance of your characters.
How useful? - 3/6 - the accessories are very useful, but you can get them elsewhere: during expeditions, at the merchant or from the pond.