Wander can travel the world on foot or by horse. To mount, simply jump upon the horse or press Triangle. Press Triangle again to dash. Do not attempt to control the horse while traveling through narrow pathways, as you can get stuck easily. Agro can find his way across narrow passages by holding Triangle.
Nevertheless, the open world found in Shadow of the Colossus serves only to lengthen the time between the next battle or to pick up additional pickups (fruit from trees - more health, glowing lizard tails - more strength, yellow points - unlocking additional "concept arts" that can be viewed in the main menu). You can also find altars that regenerate your health and checkpoints that will resurrect you in the given location if you die during your travels.
At first, you do not have access to any tools that facilitate gathering power-ups or navigating the map (Fruit Tree Map and Lizard Stone are a reward for Time Attacks). Moreover, some areas of the map are obscured and will be unlocked when you visit them (clouds on your map).
However, you can use your sword to find the colossi. Press R1 and rotate your hero to locate the area where the rays focus on a single target - this is your new objective. As always, you may encounter some blind alleys, so make sure to use the map found in this guide. The sword works only in sunny areas (when the rays are blue) - yellow rays can be used to navigate but only when you are nearby your target.