I dislike running or playing in an RPG session that is arbitrary. Oh, sure, I have nostalgia for the old Gygax tables but I used them for ideas to keep things from being stale. I would immediately find a way to make the idea logical, or internally consistent (verisimilitude - there I said it again). I will not say I am looking for realism because I am not. There is nothing real about magic missiles or displacer beasts. However, used in the proper context magic missiles and displacer beasts can seem perfectly natural in the given context. In other words, players do not suddenly go, "WTF! Where did that come from?" Rather I want them to go, "Ah, I see how that fits. Man why didn't I see that coming!"
This is not a revelation to me, this has long been my personal quest to make the games I am running be internally consistent, so that the laws of nature, magic, & gods in that environment work and the players can expect actions to have relatively expected reactions. (you can see this theme in some of my other posts). What made me feel like spouting again was my recent rereading of Poul Anderson's Three Hearts and Three Lions. (see below). One of the many things I like about the book, even though it is fantasy, there is a logic at work and you can expect the heroes, villains and monsters to all live within that logic.
First there are the humans, who can be good or bad. The best of the humans are paragons of virtue and the proponents of law. Second there are the Fay, who are typically capricious and self serving. The Fay are the primary proponents of chaos, that is when they feel like it. Lastly there are the monsters, while you are not always sure where they come from you are clear they are on the side of chaos, when they feel like. The power of the holy Christian God, and the associated artifacts, is proof against the Fay and the monsters. However, men being weak willed beings, often behave in a way that puts them in a graceless state and therefore susceptible to the Fay and monsters. A human in the state of grace, who cannot be readily harmed directly by Fay or monsters can of course be harmed by another human who has chosen to support chaos for their own selfish purposes.
In one paragraph that largely sums up the internal logic of the creatures in the world. I find it elegant for this story. I would not port this exactly for a game environment, there are some drawbacks in using that for an RPG. Perhaps that is another blog post some day.
In my homebrew, I try to have a reason that monsters exist rather than having them show up arbitrarily during an adventure. So there are humans, humanoids, giants and the like that live on the various islands and have their own lands and a creation myth that puts them there. I am a firm believer that humans are some of the most dangerous monsters. Dragons play a role in my creation myth, and though rare, have a place in the ecosystem and are involved in the early history of magic. Aberrations, though by their nature are somewhat random, are not arbitrarily placed in adventures but were created at some point in time by fell magic. That point in time may have been long ago or very recent. Those few that survived from long ago and bred may inhabit an area in numbers, those created recently are likely to be the only ones. Original undead are always created, either by necromantic arcane or divine powers, or by even darker rituals that turn the living user of that ritual into something beyond life. Some undead are able to continue their line via their dark appetites. Lastly there are the higher and lower planes populated with those beings of light or darkness with their own higher or lower purposes.
Where I struggle is with the magical creatures. I give in to a little bit of capriciousness with them, I suppose, because that is in their nature. To some extent I have the creation myth that covers the internal logic of their existence, but I try not to overdue it. I try to make the appearance of these be special and unusual and by that, in their own way, fantastic in the old world definition. So I assuage my struggle with this logic by admitting, in the end, this is a fantasy RPG.
I do not dare to compare my little world to Poul Anderson's, but I hope in some way that my monsters feel like they belong and are willing to hang around. That is until the characters kill them and take there stuff.