This fight is a little different from the others, and sees you complete certain actions. When the cut-scene is over, quickly jump on the toppled tower next to you and make your way through the ruins to the nearest standing tower. From there, shoot the bow at your opponent to make it hit the tower two times.
The towers form a type of a path. You need to repeat these actions until you've reached the final tower. These will fall down, unlocking a new area of the ruins. Continue by climbing on the roof, which enables you to use your bow. In this case, the roof will fall down on your opponent's back and destroy its armor, which protects the vital spot.
The only thing left to do is approach any of the toppled towers and provoke the colossus to attack you. Once the boss hits the tower and becomes stunned, jump on its back. The beast's current state enables you to land 2-3 charged hits in a vital point on its back. Repeat the sequence until you've defeated the boss. You can also try to stay on its back after the strikes, which is doable, but the opponent will constantly attempt to shake you off.