Do speaker cone materials matter?

We hear about all sorts of fancy names for cone materials but how much do they matter?