A Few Notes on Problem Solving

Someone asked a question about how to get better at problem solving. It got me thinking: how does problem solving work? It seemed like an interesting question to try to answer. I’m going to gear this post toward solving technical problems, but some of this may apply more broadly.

What is the problem?

The first step is to clarify what the problem actually is¹. How will you know for sure if you have solved the problem? Are you solving the right problem?

You might find some of these activities helpful in building your understanding of the problem.

• Ask questions. If you don’t know where to start, start with the very basic questions. “Why is it important to do X?” or “What is a Z?”.
• Write down or verbalize your current understanding. Forcing yourself to create concrete words is a great way to uncover things that you don’t know. You can also share this definition of the problem to see if others agree.
• Look at the details of the requirements and the situation. Small details can sometimes have a big impact on what the solution looks like.
• Learn more about the area the problem is in. If you have a problem with a database operation, you’ll need to understand the database and the code that calls it.

The real problem is sometimes different from how the problem is initially presented. If someone can’t log in, maybe the problem isn’t with their password. Their internet connection might be down. Don’t take the description of a problem at face value. Focus on the symptoms, not the diagnosis you are given. Do you own digging to uncover the core issue.

Clarifying what the problem is doesn’t only happen as the first step. As you learn more details, find possibilities, and solve sub-problems, you may find that you have new questions to answer about what the problem actually is.

Pattern matching

It can help to see if there is a similar problem that you already know how to solve. Perhaps you can even adapt that solution to this problem, or at least re-use some techniques. Knowing how to bake bread isn’t the same as knowing how to bake brownies, but you can see how some practical knowledge would carry over.

However, it’s possible to fall into a trap of getting too focused on the solution to that other, similar problem. Reshaping that problem’s solution to fit your problem doesn’t always work. Trying to force it to work can distract you from spending time working in the right direction. I see this sometimes when I give technical interviews. Candidates can latch on to algorithms they’ve memorized and try to force them to work for this problem, even though the algorithms solve a different problem. This rarely leads to anything that works.

You can sometimes get mileage out of adapting ideas or ways of thinking from other problems that, on the surface, seem very different but share some deeper patterns. The solution you draw inspiration from might not even come from the same domain.

Breaking a problem down

Solving little problems is easier than solving big problems. You can get multiple little problems out of one big problem by breaking that problem down. If you can keep breaking a problem down into smaller and smaller pieces, you might be able to make the pieces small enough that they are trivial to solve.

Sometimes you can break pieces off of a problem to make it smaller. Can you forget about file parsing for the moment and just focus on solving the problem once the data is in memory?

Other problems can be broken down into steps or stages. If you know that step A has to happen before step B, you can solve the problem of how to make step A happen as a separate sub-problem from how to make B happen. For example, if you are trying to fix a latency issue, problem A is finding where the latency is coming from and problem B is finding a way to make that part faster.

There are many different ways to break down problems. Some techniques are more useful than others for a given problem area. It helps to watch how experts in an area break down problems. Do they break it down by areas of ownership? By layers in the system? By use-case?

Solving a smaller problem

Solving a smaller problem is related to breaking a problem down. Instead of splitting the problem into separate pieces, you create a version of the problem that’s easier to solve.

Some examples:

• Instead of solving the problem for all users, how would you solve the problem for one user, by hand?
• Instead of solving the problem for arrays of any size, how would you solve the problem for an array with a single element?
• How would the function compute this if we didn’t care how long it took?
• Can I figure out how the bug happens for just this one user?
• Instead of changing the whole company’s culture, how do I get one person to do things differently once?

Solving the smaller problem won’t give you a solution to the whole problem - in fact, the solution you come up with may not even be the right direction to be going - but it will get the wheels turning and give you some new ideas.

You may find that while the original problem isn’t solved, the smaller problem has been solved before. It may work to adapt those solutions and scale them up to solve the entire problem.

There can be an art to creating a toy version of a problem that is meaningfully easier to solve but still gives useful insight into how to solve the whole problem. It seems to depend on creating a good, simplified mental model of the problem². Once you have a good mental model, you can start building a smaller version of the problem that still conforms to that model.

If your problem is cleaning up the kitchen, one of your mental models might be “dishes need to go in the dishwasher.” Using that mental model, you can create a smaller problem: putting a single dish into the dishwasher.

Smaller versions of problems are also useful for mentally testing out solutions. If a solution doesn’t work for the smaller version, it likely won’t work for the original problem either! It might be hard to see how a group would react to a policy change, but it’s probably easy to figure out how Alice and Bob will react. If they would feel insulted, it’s a good bet that others would as well.

Iterating

Theorizing all day without actually testing your theories will get you nowhere. You don’t expect to scale a mountain in a single step, and you shouldn’t expect to solve a hard problem in a single flash of brilliance. Both must be done one step at a time.

Iteration in problem-solving is a sequence of steps that alternate between movement and learning/thinking. Each thing you learn informs your thinking about how you move next. Each thing you try has the potential to teach you things you didn’t know.

When building something new, you can start with a small MVP. You build this based on your current theory about how it should work. You then try it out to learn how it actually performs (can people actually use the MVP?). This step gives you information to update your theory about how it should work. From there, you can take another step forward and build the next piece or fix the next problem. You then test that, build a little more, test again, etc. The feedback you get at each iteration will show you things you wouldn’t have thought of had you attempted to design every detail of the end state in a vacuum.

If you are solving a coding question in an interview, you might iterate by thinking of different approaches to solve the problem, then thinking about how well that approach would work for the problem. You might also iterate on different ways to break down the problem. Check your idea about how to break down the problem to see if it yields sub-problems that are easier to work with. (Does the problem get easier if I separate the problems of keeping track of which nodes to visit next from the problem of keeping track of the nodes I’ve already visited?)

When debugging, iteration is a sequence of forming theories and testing them. Your theories might be as simple as “it gets to this line in the code.” The best theories to test are the ones that have a 50/50 chance of being true. That way, each test carves away the largest possible amount of the remaining possibility space (or, to put it in information theory terms, it gives the most bits of information).

Since iteration is valuable, it behooves you to find ways to make iteration cheaper/easier/faster. All else being equal, you’re more likely to find a solution if you can try 20 things instead of 2.

Don’t fall into the trap of blind iteration. Trying a bunch of things randomly is not a fast way to solve problems! Each iteration cycle needs both thought and motion.

Understanding more deeply

Let’s say you have a car that won’t run properly. Maybe the power steering isn’t working. You know the basics of how a car works, but you aren’t a mechanic. When you pop the hood, you can point out the engine and the oil dipstick, but that’s about it. Do you think you’d be able to find what is wrong with power steering? No, you’d need to know much much more about how that part of the car works. You need to understand how the car works more deeply.

The same applies to any kind of problem solving. If there’s a system, you need to understand how that system behaves. If there’s a component, you need to understand why that component is there. Learn the gritty details. Don’t just learn that two components have a gasket between them, also learn why gaskets need to be there and what they look like when they’ve failed.

Solving problems is all about making changes to get the outcome you want. This requires understanding two things: what changes you can make, and what the consequences of those changes will be. Sometimes it’s hard to know what changes can be made, other times it’s hard to know what the results of making a change will be. Learn more about the system until you can confidently do both.

How do you understand things more deeply? Reading and experimenting are both good things to try. Read documentation, code, blog posts, bug reports - read anything that improves your mental model of the system. Experimenting can teach you all kinds of interesting things that you might never learn by just reading. The voltage across two wires might be too low, or the function might return an error when you don’t expect it. Call functions in a REPL or short script to see what they do.

Writing down an explanation of your current understanding is a great technique. It gives you a reference to use years later when you’ve forgotten all the details. More importantly, it will help you get the problem solved by forcing your brain to process the details in a more careful order. Seeing the words on a page can also help reveal questions you haven’t asked yet.

Collect facts that you’ve discovered to be true, even if you don’t know yet how they are useful.

Checking assumptions

Incorrect assumptions can be the difference between a problem that’s solvable and one that is impossible. Checking your assumptions is especially important when debugging.

If you think a piece of code didn’t run, it’s important to confirm that it actually didn’t before you go trying to find the reason it didn’t run. You won’t get anywhere trying to fix a problem that isn’t there.

It’s really easy to not notice what assumptions you are making. When you feel like you are getting stuck, take some time to step back from the problem and ask yourself what you’re assuming. Testing those assumptions is never a waste of time. Either you prove that you were right (and now you know more facts), or you realize your assumption was wrong and this saves you from continuing further down the wrong path.

Finding alternate angles

This is almost the opposite of trying to understand the situation more deeply. When trying to solve a difficult problem, it’s possible to get too focused on going deep into one detail. You can miss the fact that there’s another way to solve the problem that doesn’t even involve the detail you are staring at.

This is a good step to take when you already have the problem completely loaded into your head and the direct approach to solving the problem hasn’t succeeded. Take a step back from the problem and do something else for a while. Take a walk, write an email, whatever. Your mind will keep working on it in the background. When you come back to it later, it might be much easier to see different ways of looking at the problem.

“Rubber ducking” - the process of talking through a problem out loud, explaining it to someone even if no one but a rubber duck is there listening - is a great way to find things you’ve missed, avenues you haven’t tried yet, or assumptions you can question.

For example, you might ask yourself if there is a way to improve the system that leads to the problem instead of fixing the problem itself. If water is coming into a boat, you can either work on bailing it out (fixing the problem) or patch the hole that’s letting water in (fix the system leading to the problem). When a system is causing a problem, sometimes no amount of effort attacking the problem itself will do more than temporarily fix the issue. As long as the system is unchanged, the problem will just come back. I recommend reading the book Thinking in Systems: A Primer to learn more about this.

Experience

Solving a new kind of problem for the first time is hard. Solving a second problem of that kind might be just as hard. But over time, solving a kind of problem will become easier. When someone solves a problem much more easily than you, it’s probably because they have solved many similar problems before. Raw intelligence is a poor match for experience.

It’s really useful to watch how people with experience go about solving problems. What questions do they ask? What do they look at? What do they try first? What do they filter out as irrelevant?

I’ve found it useful to ask myself what someone I know who is an expert would do if they were faced with this problem.

Persistence

Sometimes, the best way to solve a problem is to just keep working on it. Even if you don’t see any line of attack, continue to roll the problem over in your head. You might be surprised what new insights you get if you just let the problem stew for a while.

If a problem was easy to solve, it would probably have been solved already. Don’t be discouraged when a problem feels difficult to solve. That’s what problems are supposed to feel like! Agree with yourself that you’ll keep investing units of effort (pages read, theories tried, minutes spent writing) regardless of whether they feel like they are helping.

Go solve problems

Remember that solving problems requires a mix of thinking and taking action. Don’t get stuck doing just one or the other. You have to both build and test your mental models. Writing things down will improve your thinking. Persistence is important, but sometimes the right way to persist is to take a step back and let your brain chew on it for a while.

I hope this is useful the next time you come across a problem to solve.

---

1. Assuming you’re sure the problem actually needs to be solved. Sometimes it doesn’t!↩

2. Look at the mental models here for inspiration: Mental Models and Mental Models I Find Repeatedly Useful.↩