How do I gauge how many frames per second my games are running at?

One method would be to use the freeware TSR utility, Fraps. This sits in your task tray and works in the background while you play your favourite games. The current frame rate is appended to the corner of the screen so you know at all times how well your kit is performing.

As an added bonus it is capable of capturing screen shots and saving movies of game play with the click of a button.

It works with both OpenGL and DirectX software.