On SNL from 1977-80 he created the cheesy lounge crooner, Nick, and other lovably smarmy characters. He then starred in two of the top-grossing comedies of the 1980s, playing a woolly-headed groundskeeper in Caddyshack (1980, co-written by Douglas Kenney) and a slick-talking investigator in Ghostbusters (1984, with fellow SNL alumnus Dan Ackroyd). In the 1990s Murray's comedy hits included Groundhog Day (1993, with Punxsutawney Phil) and the Amish bowling story Kingpin (1996, with Randy Quaid). He also took more serious roles, playing a mobster in Mad Dog and Glory (1993, with Robert DeNiro) and an eccentric businessman in Wes Anderson's Rushmore (1998).

His portrayal of Herman Blume in Wes Anderson's Rushmore brought him the New York Film Critics Circle, National Society of Film Critics, Los Angeles Film Critics Association, and Independent Spirit Awards for Best Supporting Actor. He has twice been nominated for a Golden Globe Award, for his performances in Rushmore and Ivan Reitman's Ghostbusters. Murray is an avid golfer and a particular fan favorite at the annual AT&T Pebble Beach Pro-Am.