The notoriety level defines how committed the guards are to find Connor. It increases when you engage in an open combat, kill them in public, steal from them or just act suspiciously (run between rooftops or push people). You current notoriety level can be seen in the lower left corner of the screen, right beside the mini-map.
There are four notoriety level which in the ways enemies behave:
When incognito, enemies don't pay attention to you unless you behave suspiciously.
Guards will keep an eye on you and eventually approach you to make sure whether you're wanted.
Enemies approach your on sight and attack afterwards.
Enemies attack you on sight and the number of patrols increases.