In the Canon:-
Broadly speaking, the generic name for all the suburbs, businesses, streets and homes that have overspilt to the other side of the current city walls. The visual evidence from The Streets of Ankh-Morpork is that overspill development has mushroomed all around the City, save on the seaboard side... as Stephen Briggs has said How much of the city should I show? A million people live in Ankh-Morpork. A walled city big enough to hold them all would be uninteresting, given that much of the action takes place in the Shades or the main civic areas. But all walled cities, especially in times of peace, had suburbs growing up outside their walls. That's how cities grow. In the books, nothing much happens outside the walls; as Terry said, I realised I didn't have to map the whole of New York, only Manhattan".
(Stephen Briggs, quoted from the pamphlet accompanying The Streets of Ankh-Morpork)
New Ankh, then, is the rest of "New York", but we really don't know how far into the state it extends or whiether it crosses the state boundary into New Jersey. The Mappe just shows the beginnings of unnamed streets on all sides, shading into obscurity: and apart from occassional tantalising mentions of people or organisations with names like the Ninja Morrismen of New Ankh, Terry hasn't really said anything much yet.
Some businesses, such as Mr Harry King's recycling plant and compost heaps, necessarily have to exist outside the City walls, not only because of the space they take up but because of the inevitable smell. The books are vague on location, suggesting Mr King's profitable but noisome business concern is located in New Ankh, both upriver and upwind. This suggests a location somewhere Hubwards, outside either Least Gate or Water Gate. Still, it's fertile turf for fan fiction.....
Speculations by Pessimal for the Fanon:-
There is a clear inference in the books that Harry King's recycling facilities, which are noisome even by Ankh-Morpork standards and which require a large amount of free land, are situated both upwind and upriver of the old walled city. This makes them the first location outside the old walled city to regularly appear in the books.
A suggestion for their location is that these are situated Hubwards of the city, and are accessed via either Water Gate (for river-borne traffic) and Least Gate (for honey-carts and similar leaving by road). There is a reference somewhere to massive compost heaps situated close to the river where garden-minded citizens may fill a sack for tenpence.
A character called Gerald Leastways appears in Night Watch. It is possible that he derives his name from a pedigree drawn in the Least Gate area of the city. Could we speculate - although this is only a supposition based on an inference - that where Pallant Street leaves the old walled city at Least Gate, when it enters New Ankh, it might well change its name to Least Way? (that is, the street and district on the other side of the wall from Least Gate.) Ankh-Morporkians being nothing if not prosaic, "Leastways" would then become a descriptive name for one born and brought up in the area.
This may become a template for all the city's main roads as they pass through the city wall - which at the time of writing has not been superceded by an even larger and longer city wall further out. (It is probably the case that Ankh-Morpork, like the European cities it is based on, has reached a stage where building walls around itself has become obselescent, having already outgrown at least two which are now swallowed up inside itself as growth becomes inexorable.)
We therefore might expect the continuation of Kicklebury Street as it passes through the Limping Gate to become, perhaps, Limping Street or Limping Way. Upper Broadway starts at the Un-named Gate: its continuation on the other side might be Sto Helit Way, or the Helitian Road (known to citizens as the Road to Hels?) Losing Street, inside the city, begins at the Onion Gate. Onion Road?
Pigsty Hill leads to the Shambling Gate - only right and proper, as this is the drovers' route from the Chalk Country to the Shambles, the slaughterhouse district. Therefore the continuation road from the gate is Drovers' Way? And given that drovers with large herds need to be able to stable them over night, especially if they arrive after the city gates are closed, this area of New Ankh might be called Droversfields?
Dimwell Street begins at Traitors' Gate. Upper Dimwell? New Dimwell Road?
Park Lane/Edgeway Road lead to the Rimward Gate. Rimwards Road?
King's Way/Dark Gate/King's Down/ Deosil Gate (in order from the river outwards) are the a recognised road for Quirm. If we use a little Quirmian and suggest the original name might have been 'Rue Des Rois", what about "Ruddyroy Street" as a local corruption?
The only caveat to the idea of city walls and fortifications becoming obselete and more expensive to build as the city gets larger and the line of wall longer (expensive to build, had to man, expensive to maintain and vulnerable to artillery) is that in the 17th and 18th centuries, many European cities adopted the new defensive philosophy of geometrically designed "star-forts" at strategic locations. These could safely be left unmanned and only garrisoned if there were real and pressing need. Those which needed a permanent garrison also served as useful barracks to keep the rough soldiery out of the way of citizens.
The principal castle of the Tump looks to be vulnerable and open to concerted attack from the north and west. What barbicans, skirting walls and other additional defences might have been built in to cover this weak spot - such as the star-forts of a later age?
What mediaeval city walls evolved into:- http://en.wikipedia.org/wiki/Star_fort