Monsters do not necessarily need to understand pathing and collision with the world to be in a confined space. There are plenty of tricks and hacks with spawn regions and how you lay out an area that will prevent mobs from running through walls or doing the impossible to reach the player.
The "dungeon" does exist in-game. Right before the sale David teleported probably 100 people from both Chaos and Order to see what was being worked on before Tulga was forced to cease development. Only the physical layout, the atmosphere effect, and some world objects existed at the time, but it was impressive nonetheless.
I ran around the "Blighted Labyrinth" a good while before the sale while I still had access to Genesis. I was sure to take plenty of snapshots along the way as well. Here are a few to look over,
be sure to click them for a full-sized version. Ignore the red lines you see, they were a by-product of being logged in to the development server with regions set to be visible.
Entrance to the labyrinth
Undead florae
Bubbling pool with rock bridge
Hidden passage on the left (look closely)
Undead plant with petals open
Prototype scripted sequence w/ caged Amadan
Maggots lurking
What's sad is that this area will probably never see the light of day as it was intended. There were to be several new monster types along with chests/mimics hidden behind narrow secret corridors and other clever little paths hidden from view. I found four or five places where wandering off the beaten path would lead you to a secret passage with what looked like a chest spawn area. From what I understood players would need to craft keys to open the chests, and there was always the possibility a mimic would spawn instead of a reward.
None of the monsters were fully complete to my knowledge, and getting the keys to work required simulation overhaul that only Smeglor really knew how to do. There was also a planned scripted sequence with the caged Amadan involving a NPC teleporting in, walking around, having a conversation with the prisioners, and I think interacting with the player and maybe casting a spell or two.