In Riven I think you can do it by pressing Alt, and choosing Windowed mode in the menu that appears at the top of the screen.
In ScummVM you use Alt+Enter to switch.
In RM:ME the option is in the interface - look in the settings by pressing Esc. Not sure if realMyst has the option, but if it doesn't there's probably some hack to make it windowed.
In Myst 3 and 4, you can't do it. I've tried to do it, but starting the game windowed completely messes up the mouse. You could try a Virtual Machine if you absolutely must.
By the way, I think wondowed is the single best spelling of the word windowed I've ever seen.