This page of The Elder Scrolls V: Skyrim guide includes all the important information about casting spells. In addition, we have prepared a complete list of spells of the school of Illusion, Alteration, and Conjuration. The action of each spell is described in detail in the table.
There are dozens of magical spells in Skyrim you are allowed to cast and they represent five different schools of magic - illusion, alteration, conjuration, restoration and destruction (more info on each school can be found in the Character development chapter of this guide). You start the game with a very small number of basic spells and it's up to you to get your hands on the rest of them. Don't think that you'll be receiving new spells almost automatically, because only a small percent of them is unlocked as a result of completing quests. All other spells must be learned by reading spell tomes and they can be obtained on two different ways. A much easier solution is to find a merchant that specializes in selling spell tomes (screen above) and the quickest way to do that is to visit all the mages which live in College of Winterhold. Sadly almost all spells tomes cost a lot of gold. As a result you should consider solution number two - finding (or stealing) spell tomes while exploring the world. Most books can be found in the cities, villages and forts, however some of them (including rare ones) are hidden in dungeons.
Once you've obtained a spell book you must learn the spell it'd about. In order to do that you must start by opening the inventory. Go to the books section, hover the mouse over a spell tome you're interested in and press the left mouse button. Notice that reading a spell tome results in the main character losing this item, so you can't get your money back (if you've bought it). Each spell can be learned only once and it never disappears from the spell list, so if you find a copy of a spell tome you've already read you'll be left with a choice of leaving it alone or (better idea) selling it.
Each spell has its unique properties, however two of them are very important - spell level and mana cost. You can find these informations by choosing a spell from the list, hovering over it and examining the text seen on the left side of the screen. A spell's level is important if you want to get serious with magic and if you're unlocking perks from the skill trees. This is also a good idea to find out how powerful a spell actually is and obviously over time you should focus on switching to the best types of spells - expert and master. As for the mana cost, this is a number of mana points you need to spend in order for the spell to be casted. Obviously better spells need a lot more mana and if you don't have enough of mana points you may even be forbidden from casting the spell until you increase your mana pool. Make sure to check if the spell needs mana only once or if it's a sustained one. The second kind of spells is much worse, because they start to use mana as soon as they're activated and they don't stop until the main character has stopped casting a spell or (more likely) until all mana reserves have been drained.
An alternative to casting magic spells is using magic scrolls. They're obtained in the exact same way as spell tomes, so you can purchase them from merchants or find while exploring the world. The advantage of scrolls is that they don't require any skills in order to work. The biggest disadvantage is sadly far worse, because each scroll can only be used once and then it's lost forever. Investing in new spell tomes is as a result a much better idea, however if you manage to find a scroll you shouldn't sell it or get rid of it. It's best to activate it during a particularly challenging battle.