The College of Winterhold is a place uniting those who can use magic, therefore serving as a type of a Mages Guild. If you want your hero to specialize in magic, you should of course visit the College early on in the game. The advantage of such solution is that by solving quests and interacting with members of the College, you will be able to increase your skills faster and gain access to precious equipment. If on the other hand you don't intend to use magic at all, or in a very limited range, you can postpone you visit to the College, especially that it's connected with the Elder Knowledge main quest. What's interesting, by doing so you will be able to enter the College without any magical skills.
The quests of the College of Winterhold include both main as well as a few side ones, connected with exploring various areas of this location or helping the mages inhabiting it. They can also train you in different magic types, therefore speeding up the process of reaching higher experience levels. The College itself is in the north-east corner of Skyrim and is accessible through a long bridge, connecting it with Winterhold (screen above). After the initial conversation, you will gain access to most of the College rooms and only the Arch-Mage's Quarters will be locked. Additionally you will be free to use the room that you receive, including the bed.