January 12, 2014

I just finished reading a great and insightful book, Dogfight, by Fred Vogelstein. The book chronicles the fight for dominance in the mobile computing market between Apple and Google. It’s extremely well written and a good page turner.

What interested me most about this book was the level of chaos the two companies went through writing the code to make the iPhone and Android devices. I went into the book believing that the smooth presentation Steve Jobs made when he unveiled the iPhone was a reflection of how together and polished Apple is as a developer. Being in the same sort of business, on a much smaller scale, I envied the apparent calm and organized development process a company like Apple or Google goes through to bring out a cool new product.

Of course nothing could be further from the truth. It was utter chaos coupled with a lot of nail biting right up until the moment the phone shipped out the door. And while our own development process also involves a flurry of activity customers never see, it pales in comparison to what the big guys go through.

When you pick up your shiny new product, whatever it may be, I think it’s good to know that a healthy chunk of people’s lives and passions went into its creation. That knowledge helps us relate to the soul and life of the machine.

Subscribe to Paul's Posts

8 comments on “Dogfight”

  1. I am an audiophile (of some sort) and I also happen to have a degree in computer engineering. In the past I’ve worked for a computer giant developing digital chips. Get a PC motherboard, pick some chip. Then imagine that I was part of a ~20 people team working on **A TINY PORTION** of that chip. Well, a tiny portion of a tiny portion would be more accurate. Idea-to-market time is measured in years. Years of tough intermediary deliveries, thousands of nasty bugs etc. Weekends spent at work and all that.
    I don’t want to be condescending or anything, but by comparison audio seems child’s play.
    During a private conversation with someone from management, the guy asked me “what do you think we’re doing here?”. I said, “well, we’re making a bug-free product.”. He smiled and answered “no, we’re hoping to be able to deliver a product which the customer will accept”. Note that it doesn’t include “bug-free” 🙂
    So don’t go reading books on how airplanes are made 🙂
    It’s hard to grasp the complexity of computers nowadays even if you’re an EE, let alone a complete outsider. You really don’t get a taste of it until you’re directly involved.
    If these guys were to be perfectionists to the last, tiniest detail and aim for perfect planning, a new PC would come out once in 30 years and would cost a few times the price of a very good car. There are smart phones out there that are faster that the laptop I’m typing this message on, and it’s a pretty new one. It proves that the business model actually works in real life. Absolute perfection only works for very small scale projects. In fact, I think it rarely works there too 🙂
    Also, I would add that the classic image of the engineer, buried under oscilloscopes and all that, is a thing of the past. Nowadays real-life engineering is more like technical bookkeeping. Meetings, communication skills, willingness to travel for more meetings, bureaucracy, etc. Corporations are very bureaucratic organizations. Not exactly “romantic” places, if you will. It’s not terrible but not the greatest thing either. There’s a decent living to make in the field, but if you’d rather spend time with scopes and multimeters instead of participating in endless (and not rarely pointless) meetings, well…
    So, now you know 🙂

    1. Thanks. Yes, I am well aware of the huge differences but the spirit of my post was to scale the two extremes in such a way as to show that modern programming has an element of chaos that is the same in both cases, that chaotic element rising exponentially with the complexity of the program. But it’s there in both.

      There is no such thing as bug free programming when the level of complexity exceeds a certain tipping point. You then accept that the best you can do is make it so the bugs are not noticeable to the user as best you can.

    2. For 16 years I was on the front lines of the monumental effort to turn what’s come to be known as the legacy copper telephone network system at ATT&T’s divestiture in 1984 into the nascent internet of everything we continue to evolve into today. I’m not going to bore you with endless war stories but you are right, the level of complexity is mind numbing even for electrical engineers. The war between Apple and Google is a small skirmish, one I think Google will surely win. Even Jobs’ original partner Steve Wozniak said at Jobs’ eulogy he wished his iphone could do half the things his Android phone can do.

      Much of what I worked on and still work on today are termed “mission critical” which means they cannot fail. Not just profits but the survival of businesses, even human life can be at stake if there is a technological failure. For example, some of these systems are used by hospitals in every area of treatment including on line in real time in operating rooms.

      IMO industry has made a fatal blunder far worse than the total mismanagement in the financial industry. The consequences might also be far worse. There is an inherent contradiction between reliability and cost. You cannot have both. When business management types replaced technocrats in hands on managing of complex technical systems they sowed the seeds of the eventual failure of these systems. Reliability is like an insurance policy. You buy it hoping never to have to collect. Standby equipment, systems, and reserve capacity are expensive but when needed even if rarely they can be the difference between continued operation and system failure. Not only won’t managers spend any more than they have to and no sooner than they have to on anything including capital maintenance, expansion, and replacement, outsourcing to competitive bidders has meant a dwindling of skilled personnel in some critical technical areas. Outsource consulting firms cannot afford to train next generations of them. When those with these skills retire, die, go into other areas they are not replaced. When they are needed, there won’t be enough available. They cannot be trained on short notice and they cannot be imported because systems, standards, products, terminology even basic design philosophies are unique to the US. These are skills that require a lifetime commitment to develop.

      One ominous consequence of this mismanagement and the growing coalescence and complexity of networks is that when they fail, they can fail in what is called a global catastrophic network collapse mode. It means the whole thing goes down like a house of cards. This can happen on a small scale such as a company’s data center or to a whole power grid. It happens. It happened in October 1965 when we had the Northeast power blackout resulted from a single transformer failure in Niagara New York. It happened 39 years later in August 2004 when the entire Northeast quadrant of North America blacked out as a result of a single feeder in Ohio failing. In September 2011 5 million people in the Southwest US and Northwest Mexico went black when a single piece of equipment was taken off line in Arizona. We do not know how to effectively predict, contain, or prevent these kinds of failures but we do know that the risk increases exponentially as the systems become more complex and are taxed to their limit often as the result of cost cutting. The consequential damage may be far beyond the equipment that initiated the failure itself. Enjoy the video linked below. These circumstances may be unique but the principle isn’t, it’s universal.

      1. Hi

        Incidentally, I also worked in a field that they called “safety relevant”. Where bugs could mean that people get killed. It was a much more rigid process compared to the company I described above, the smallest update was long analyzed etc. But that didn’t seem to actually make an improvement in bug rate. It proved that when overregulation is enforced, people will eventually find workarounds to very tedious and bureaucratic tasks. I think that what gets a job done well is the human factor: good engineers wanting to get a good job done. Regulations are just that, regulations. They don’t get the job done. Also, I think that overregulation creates another problem: some people forget why it was needed in the first place and end up doing it just for the sake of it. We are accustomed to it, so it mush be a good thing, right?
        I remember a Dilbert strip where his boss was telling during a meeting that they’re hoping to devise a process which after refinement will eliminate the human factor altogether 🙂 Artistic exaggeration aside, that’s exactly what seemed to happen in the latter company I mentioned: they were trying to compensate for the quality of the human factor by a very rigid, almost paradoid process. Which, IMO, is impossible. It’s funny that you get to find that such things are universal from a comic strip.
        IMO both models actually work and when it comes to the end result they can get the job done or not.

  2. I can sympathize with complex things taking some extra time to market, like a new computer, phone, or amplifier that the designer has to make right (or at least right enough per B77ng). What is hard to understand sometimes is a manufacturer that has missed 2 or 3 deadlines so far on a powder coated steel wall box with 4 flip out dog legs. Not ever having been in the manufacturing business at all, I have heard some tales of how expensive a UL listing can be (per product) even on a line of similar model low voltage speaker back cans. It is my understanding that the manufacturer must purchase this UL listing for each model can just because it’s sizes are different. I have also heard that most if not all the products that I sell are built in China and often times the first renditions have not been built to proper specs (ie a multi- room controller/amplifier that shows up to the manufacturer with cheaper components than specified. We on the outside probably have very little idea what it takes to get to market, even on a steel wall box. But I still grumble and bitch about it anyway.

Leave a Reply

Stop by for a tour:
Mon-Fri, 8:30am-5pm MST

4865 Sterling Dr.
Boulder, CO 80301

Join the hi-fi family

Stop by for a tour:
4865 Sterling Dr.
Boulder, CO 80301

Join the hi-fi family

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram