What do you mean by "locked room"? Are you talking about a vault? Vaults can hold some nasty buggers. If you have Track or some other means of detecting monsters, always use it outside a vault. If not, be really, really careful and always have a method of escaping quickly -- blink, teleport runes, movement infusions, or a class/generic talent. You got really unlucky here, granted. An enemy more than twice your level is rare. But it happens. Sounds like you were in Nur, since that's the only place I can think of that a 17th level player usually visits that spawns boiling horrors. It's a nasty little dungeon, and a 17th level character is going to be in a lot of danger there.

https://te4.org/wiki/Vault

Glowing chests, tentacle trees and demon statues can spawn out of depth monsters too.

That's a risk you take playing ToME4.
As a noob player I have 1 winner and hundreds of loosers.
Is gatekeeper in Sher'tul OP? YES! Am I angry? Yes, I ... was. For first 30-ish times.
Wanna kill trader's wife in crypt and died as she is a OP necro? Haha. Shame on you.
Just accept it, Neo. You can't control FATE.