À propos the previous post, wanted to share a 2019 article by Gregory Travis about the two Boeing 737 Max disasters that shook the world in late 2018 and early 2019. It’s titled How the Boeing 737 Max Disaster Looks to a Software Developer, and I found it fascinating, well-written, and filled with great details.

The Max variant of the 737 was redesigned to allow larger, more efficient engines, but this made it less predictable, which the company decided to paper over in software:

The airframe, the hardware, should get it right the first time and not need a lot of added bells and whistles to fly predictably. This has been an aviation canon from the day the Wright brothers first flew at Kitty Hawk.

Apparently the 737 Max pitched up a bit too much for comfort on power application as well as at already-high angles of attack. It violated that most ancient of aviation canons and probably violated the certification criteria of the U.S. Federal Aviation Administration. But instead of going back to the drawing board and getting the airframe hardware right (more on that below), Boeing relied on something called the “Maneuvering Characteristics Augmentation System,” or MCAS.

Boeing’s solution to its hardware problem was software.

This feels like an interesting extension to the fly-by-wire story of Air France Flight 447, where Boeing seemed to be on the other side of the fence to the Airbus. Perhaps no more:

[…] It seems that with the 737 Max, Boeing has changed philosophies about human/​machine interaction as quietly as they’ve changed their aircraft operating manuals.

Travis argues that people building the software did not fully understand the shift, and particularly didn’t really understand how pilots fly the airplane during more challenging moments:

Those lines of code were no doubt created by people at the direction of managers. Neither such coders nor their managers are as in touch with the particular culture and mores of the aviation world as much as the people who are down on the factory floor, riveting wings on, designing control yokes, and fitting landing gears. Those people have decades of institutional memory about what has worked in the past and what has not worked. Software people do not.

The real reason? Money.

MCAS is certainly much less expensive than extensively modifying the airframe to accommodate the larger engines. […] What’s worse, those changes could be extensive enough to require not only that the FAA recertify the 737 but that Boeing build an entirely new aircraft. Now we’re talking real money, both for the manufacturer as well as the manufacturer’s customers.

That’s because the major selling point of the 737 Max is that it is just a 737, and any pilot who has flown other 737s can fly a 737 Max without expensive training, without recertification, without another type of rating. Airlines—Southwest is a prominent example—tend to go for one “standard” airplane. They want to have one airplane that all their pilots can fly because that makes both pilots and airplanes fungible, maximizing flexibility and minimizing costs.

It all comes down to money, and in this case, MCAS was the way for both Boeing and its customers to keep the money flowing in the right direction. The necessity to insist that the 737 Max was no different in flying characteristics, no different in systems, from any other 737 was the key to the 737 Max’s fleet fungibility. That’s probably also the reason why the documentation about the MCAS system was kept on the down-low.

Put in a change with too much visibility, particularly a change to the aircraft’s operating handbook or to pilot training, and someone—probably a pilot—would have piped up and said, “Hey. This doesn’t look like a 737 anymore.” And then the money would flow the wrong way.

There are some harsh words toward the entire software industry toward the end. This feels directionally accurate to me, and resonated even before the arrival of generative AI:

I believe the relative ease—not to mention the lack of tangible cost—of software updates has created a cultural laziness within the software engineering community. Moreover, because more and more of the hardware that we create is monitored and controlled by software, that cultural laziness is now creeping into hardware engineering—like building airliners. Less thought is now given to getting a design correct and simple up front because it’s so easy to fix what you didn’t get right later.

I want to show you something glorious. This is Bear, the note taking app:

There are desktop apps that get flustered if you ⌘+Tab away and back, misplacing focus or closing a dialog box inside. There are iOS apps that fully reset themselves whenever they get swapped out of memory and have to be reloaded.

But Bear, right here, remembers which note you were on, and exactly where you were in that note, even between phone reboots.

Software is transient and malleable, and one of the hard parts is knowing when that’s beneficial and when detrimental. In real life, you can leave a notebook on your desk, open on a certain page, leave a pen pointing to a specific word – and then depart for a two-month trip to Europe. You will find your notebook exactly how you left it. Why shouldn’t software behave this way?

Also, another thought: This is very likely not something users will complain about when broken, or suggest when absent, even if you go out of your way to open yourself for feedback. Just swapping an app out of memory is hard to understand and “repro” (in engineering parlance). There’s a certain design mindset and taste necessary to notice and care, and a certain vision to carry it through.

The lack of direct user feedback doesn’t mean it’s not worth doing. It just means that there are some things that designers and only designers will know how to properly weigh, describe, and prioritize. If you have a few design-minded users that actually send you feedback like this – treasure them. But most likely this will have to come from “inside the house.”

To me, it’s clear that within Shiny Frog (the makers of Bear), there are people who care about this kind of stuff, and leadership that trusts them. Kudos.

Just like the video I shared last week, this 20-minute video by Mariana Colín at The Morbid Zoo is sharper than most, and also extremely entertaining:

Colín is not “in tech,” and the video is of “the king is naked” variety which is very, very refreshing.

Among many good observations, this caught my attention as relating to this blog’s topic:

It’s a little weird to have this almost adversarial relationship with your customer base. They’re not trying to solve a problem customers have. They’re trying to convince people that the product on offer is something more than it clearly is.

What VR is, is a fun parlor trick. What they want VR to be is literal reality.

It does indeed feel Meta’s version of VR/​metaverse has always been cargo-culting real world in a particularly awkward fashion, which Colín analyzes deeper.

Too many quotable laugh-out-loud moments, so maybe just this one more:

Down here in the real world, there are really only two things a media technology can be. It can be a solution to a specific discrete informational problem, or it can be an artistic medium. These two things are not mutually exclusive. There is crossover here – like, radio was a military tool before radio plays were ever a thing.

But by the former, I mean you’re literally just making information go faster. You’re reducing the amount of noise between a message and its receiver. Any kind of metaverse is going to be really, really bad at this because you don’t need to look at a weird Pixar version of your coworker in order for them to convey what a deadline is.

I read Erika Hall’s Just Enough Research. I’m not going review the entire book as it feels a bit off-topic for this blog, but the chapter about surveys had me nodding my head so much I’d love to excerpt a few things:

The questions can be asked in person or over the phone, or distributed on paper or collected online. The proliferation of online survey platforms has made it possible for anyone to create a survey in minutes.

This is not a good thing.

Surveys are the most dangerous research tool — misunderstood and misused. They frequently blend qualitative and quantitative questions; at their worst, surveys combine the potential pitfalls of both. […]

If you ever think to yourself, “Well, a survey isn’t really the right way to make this critical decision, but the CEO really wants to run one. What’s the worst that can happen?”

Brexit.

Hall highlights that surveys are much harder to debug than other methods:

It’s much harder to write a good survey than to conduct good qualitative user research—something like the difference between building an instrument for remote sensing and sticking your head out the window to see what the weather is like. Given a decently representative (and properly screened) research participant, you could sit down, shut up, turn on the recorder, and get useful data just by letting them talk. But if you write bad survey questions, you get bad data at scale with no chance of recovery. It doesn’t matter how many answers you get if they don’t provide a useful representation of reality. […] Surveys are the most difficult research method of all.

[…] Bad code will have bugs. A bad interface design will fail a usability test. A bad user interview is as obvious as it is uncomfortable. […] A bad survey won’t tell you it’s bad.

And that they might be seductive because they feel like hard data:

Designers often find themselves up against the idea that survey data is better and more reliable than qualitative research just because the number of people it is possible to survey is so much larger than the number of people you can realistically observe or interview. [… But] unless you are very careful with how you sample, you can end up with a lot of bad, biased data that is totally meaningless and opaque.

There’s also this hilarious bit:

Managers love NPS because it was designed to be loveable by managers. It’s simple and concrete and involves fancy consultant math, which makes it seems special. But is this metric as broadly applicable and powerful as it claims to be?

Nah.

NPS is not a research tool. I shouldn’t even be talking about NPS in a research book.

The entire book is worth a read, with a lot more to offer than the pithy quotes I excerpted above. I really liked its pragmatic approach to research that understands the realities of the industry.