New main objective: Escort Esbern to Riverwood
Wait for Esbern to gather his stuff and use this occasion to take a look around the cell yourself, finding a couple valuable items. After Esbern tells you that he's ready, save your game and decide whether you want to get out of The Ratway on your own, or ask the old man for help (screen above). It's not necessary in my opinion, as you will be going through the same corridors as before and the only difference will appear at the very end.
Be careful, as right after getting out of Esbern's cell you will get attacked by enemies, who also have been searching for him. I'd recommend taking care of the Thalmor Wizard (screen above) as fast as possible and only afterwards move to the Thalmor Soldiers. Luckily winning this battle will be rather easy, considering that Esbern won't just sit around, but actively participate by casting powerful spells.
Afterwards you can continue onwards through the already visited locations/ first you have to return to The Ratway Vaults, afterwards to The Ragged Flagon and in the end to the main part of The Ratway. In the last location, you will have to follow a slightly different route. After reaching the room where the lowlife was, choose the previously ignored northern passage. Approach the drawbridge and use the lever (screen above). Get ready for a fight with Shavari. Examine his body to find Shavari's letter, which you should of course read. Eventually you should reach the sewers exit and therefore Riften.
New main objective: Talk to Esbern
The good news is that you don't need to go to Riverwood by foot, as you can use fast travel. After reaching the destination, of course head to the Sleeping Giant Inn. Listen to a short conversation between Esbern and Delphine and afterwards head to the basement room (screen above). Another conversation awaits you there, from which you will learn of the title Alduin's Wall. Alduin's knowledge was gathered there and it might be used to slay the dragon. As you have probably guessed, the Wall will be your next destination.
In the end Delphine will want to know if you want to head to Karthspire together, or meet with all the participant there. The choice isn't really important, as even if you ask Delphine and Esbern to accompany you, you will most probably have to lead anyway (the AI can be a bit irresolute), instead of following one of them.