Once you've returned outside take your time to enjoy the fireworks and you can also listen to what Joker has to say. Your next objective will be to reach the Penitentiary building, so you'll have to travel to Arkham West. Notice that Joker's thugs are standing next to the entrance. They won't attack you, but you can attack and defeat them without any problems.
Choose a smaller entrance to the building which is going to be painted in red (screenshot #1). You'll enter a corridor where you'll encounter even more of Joker's thugs. Just as before you should consider attacking them. You'll have a chance to score a lot of points here, especially since your enemies won't be carrying any melee weapons. Once you've won proceed through a nearby door and you'll end up in the Visitor Room.