The size of your team depends on the number of military buildings. The only exception is the defense tower. Each building gives you two extra slots in your party/squad. Of course, they can be improved, which in addition to increasing the effectiveness of your soldiers, will also increase the maximum limit of your soldiers by one.
There is no upper limit on the number of units. As long as you build and improve more buildings, the maximum number of soldiers you can train will increase. You are only limited by the maximum number of buildings you can construct in your area.