In writing a formal letter, when do you use 'To whom it may concern?'

I would use it whenever I didn't know to whom I was writing.  It is best to try and find the name of a responsible person to whom you can address your letter.  It's usually not that difficult these days.  If it's a cover letter for an application I will admit I often just write, "Hello," and start that way, rather than "To whom it may concern,".  Remember for applications we often only have seconds to make an impression and hold/gain interest.