Right after teleporting to the market, move forward. Right before the castle entrance, on the left there is a building called Hauteville Heights which you have to buy . Go to the backyard and pick up the silver key .
To the second one, return to where you appeared after the teleporting here and cross the bridge to the other side . Go down the nearby stairs onto the embankment and go as far to the left as you can . Behind the crates there is the key and a treasure chest.
The next key is in the underground complex to which the door below the bridge leads . It becomes unlocked during "Crime & Punishment" by the end of the game. Once underground, move forward all the time until you reach the cells .
On the right (behind the door) there will be stairs leading to the lower level . Go down and pick up the key lying in the cell .