I absolutely have to agree with Joe.
The lacking defensive value of walls and towers is my biggest gripe about SH3 (bigger than a lck of base building AIs and even bigger than a missing "Siege that"-Mode).
Did siegers in real life clean up the walls with spear throwers and then just send their ground troops with pickaxes to tear down the castle?
No, of course not, because they couldnīt

Instead they used numerical superiorities, siege engines and earthworks.
You say that archers were overpowered on walls, Lucifer ...
yes, they were, as they were in real life.
So, why not, instead of making walls and archers weaker, do it like siegers in real life, i.e. use measures to render the walls useless, by hurling large stones at them (via catapults/trebuchets), digging under them (via sappers) or bringing enough ladders and men (maybe on a 10:1 ratio) so that the soldiers can climb the walls and defeat the archers in melee, before all troops are killed and/or they attack the gates and walls via battering rams.
This would work even for Multiplayer ...
yes, the players would not be able to attack each others walled base for a long time (that is, unless they managed to launch a successful surprise attack before the other player has built his walls.
But they would be able to do many other things, like attacking the enemy woodworkers and miners outside of the castle, with one player without doubts being able to cut off the other one from his supplies (at least with regards to wood, iron and stones) and then finally assembling an army big enough to storm the castle
(another way that would probably placiate everyone, me included, would be, to just have it as an option in the game,
meaning before starting a battle you can chose between "realistic siege mode" and "easy siege mode" (with the easy mode being the game balance as it currently is, and realistic being a game where walls really have a defensive value and you really need a large numerical superiority as well as siege engines to storm the castle [and would also mean that normal troops arenīt able to destroy walls]).
This way everyone can chose how he wants to have his sieges

)
This way (i.e. with the 2 mode-system I just suggested), you wouldnīt even have to change the campaign ... just make it easy mode only ...
the only thing you would have to change would be the historical single player siege maps (with giving the attacker far more troops or even, which is something I hope for, allowing attacker and defender to buy his own troops with a predetermined amount of gold [with the attacker getting much more gold than the defender]) and of course making the random encounter in single player base building mode much stronger