I am not a speaker designer, but I've spent much time with some of the best in the world.
I don't envy their task.
When I design electronics the initial work is relatively straightforward and the results predictable. Even first year engineers haven't too much trouble designing platforms with full frequency response and low distortion.
Once completed some of us take the extra steps of voicing our work, a process that takes years of experience to get it right.
None the less, even if you don't voice your work you're miles ahead of that which faces speaker designers, because the elements they have to work with are so flawed in the first place.
Here's an example:
This is a response curve from a ribbon tweeter. To understand the difficulties faced by speaker designers, just look at the solid green line for starters. This represents what's known as the "on-axis" response. The measurement microphone points directly at the tweeter. Yikes!
Even straight on this thing's all over the map. It's about as far away from "flat" as a roller coaster.
But, now look at the other lines. They are what we hear off-axis, meaning the microphone (like the listener) sits to the side of the tweeter. These aren't different by a small amount. Each response differs by many dB.
And as Audiophiles we worry about tenths of dBs?