The second fight with the queen takes place shortly after defeating her the first time. Between the fights, you will have to solve a simple puzzle connected to the keys that you've found. The game will allow you some time to prepare, heal the heroes and save the game.
First, you have to defeat Queen Leech the same way as you did during the first fight. You can control both heroes, but it is better to focus on Rebecca and try to remain 100% HP. In the meantime, Billy should attack the boss using the weapon he has. Help Billy only if there is no direct danger to Rebecca. After the fight, a cut-scene will be triggered and you will receive Gate Operation Manual. Now you proceed to the main part of the battle, where you can control only Billy.
Note - don't mind the time limit that you see on the screen, because you can win the fight long before the time ends.
The strategy is as follows: As Billy, you need to cover Rebecca who will travel from one valve to another to open the ceiling and thus defeat the boss. Every time Rebecca starts heading to the next valve, the boss will focus on her and will try to kill her. This is the most important moment, because you need to quickly attack the queen with Billy and deal as much damage to her so that she starts attacking you (use Magnum or Grenade Launcher). When it happens, try only to endure the boss's attacks. Don't use your best weapons on the queen all the time, because you can't defeat her this way.
As it was said earlier, Rebecca will have to reach the valves. Their location (in order) is as follows: in the south-western corner, in the north-western corner, in the south-eastern corner and in the western part of the room. You don't have to remember their location. Because Rebecca will find them on her own. The game will trigger a cut-scene every time Rebecca reaches a valve and also when she's done working on it.
You have to protect Rebecca until she uses all four valves and open the ceiling, killing Queen Leech.