Chapters 2-3 | Voxophones BioShock: Infinite Guide
Voxophone #1: Love the Sinner - Lady Comstock

Welcome Center - in one of the niches of the chapel located to the left of Father Comstock's bust.
Voxophone #2: Undeserving - Zachary Hale Comstock

New Eden Square - inside the Hudson's tailor shop, to the right of the monument of Father Comstock.
Voxophone #3: Everyman, All At Once - Zachary Hale Comstock

Parade Crossing - in the garden to the right of the flower girl encountered after crossing the draw bridge.
Voxophone #4: For I Am Lonely, Too - Constance Field

Path of the Scroll - under R. Lutece's monument found after leaving the fair.
Voxophone #5: Solution to Your Problems - Jeremiah Fink

Raffle Square - soon after ending the first battle with the policemen. The recording can be found between the cages, near the Veni, Vidi, Vigor! machine.