
Just don't forget to continue making your economy grow (as in build more town centers, farms e tc. I'd even just wall with houses, or not far around my base. Since it's a ludicrous map, it should take your oponent several minutes to reach your base and you can do this with ease. Then have 6-8 villagers on stone and research stone mining as soon as you begin advancing to Castle age. Then make a blacksmith and market as soon as you reach feudal age soon and click on castle as soon as possible, around the 34 villagers mark is good probably. Just advance with 31 villagers or so into Feudal Age, for a ludicrous map is quite good honestly. Also castles deal with towers very easily.

Build them with 5 or more villagers if you wish. My advice is that instead of building those towers, save that stone for building a castle or two to protect your main resources.
