Is there such a thing as too much power?

What happens if you exceed the rated wattage specifications of a loudspeaker? Is there a problem with too big of an amplifier connected to a speaker?