Before you go about eliminating the High Overseer, I recommend that you explore the entire of the [Office of the High Overseer] location. The ground floor consists mainly of the main room in which one of the overseers is giving a lecture (the above screenshot). If you are thinking of exploring the area, you need to wait for the lecture to draw to an end and for the crowd to disperse. You might want to have a look at the seven plates. You could also steal gold from the locker and search the area for Piero's spiritual remedy and a book.
In both side rooms there also are plates to see, but you need to be aware of the fact that each of these rooms may be guarded by a guard. I also recommend locating the door to the right of the main entrance (the above screenshot) because it leads to the basement. It is worth observing, that it is in the basement that you would be starting the exploration of the building, had you decided to access it through the Kennel.
The most important thing to do here, is interact with the bust shown in the above screenshot. This will allow you to access [Campbell's Secret Chamber]. Inside you will find gold, food, ammo, sleeping darts, a letter (My Girls), a book, a rune, Sokolov's painting oraz Audiograph - Campbell's Lamentation.
You can reach the first floor by using the stairs on both sides of the main entrance, assuming of course that you have started the exploration on the ground floor. In both cases, you need to be careful, because the staircases are guarded by single guards.
The first floor corridors are guarded by many overseers. If you want to avoid meeting them, you should make use of the abovementioned window sills and opened windows. If, in turn, you prefer to eliminate them, do not forget about concealing the bodies, or otherwise, this may lead to alarm being raised by a stray guard that used to patrol the ground floor earlier. .
New optional Quest: Render High Overseer Campbell unconscious
Two of the most crucial rooms for this mission (save the meeting room that is directly connected with the main quest) are located in the building's eastern wing. Firstly, you should go to the Archive room. Be careful though, because there are two enemies inside (one downstairs and the other one up above). After you eliminate them, interact with the book On Branding Heretics, This way you will complete a side quest. It's an important thing to do, because this way, you will learn about an alternative way to get rid of the Overseer. In the Archive Room you will also find seven other books, plus food and gold.
Another one of important locations is the Interrogation Room. In the main part of the room, you will find gold, Sokolov's health elixir and a spring razor. Go now to the control room located upstairs (the above screenshot). The area is locked away, but you can access it with an interrogation room key that can be stolen from the guards patrolling the corridors. Inside you can find gold, two books, bolts, sleeping bolts, Heretic's Brand (the thing you will be able to use to excommunicate Campbell) and an Audiograph - Whaler Assassin Interrogation).