Skip to content

Simon Belak explains what tech job interviewers are really looking for

In Meaningful work interviews I talk to people about their area of work and expertise to better understand what they do and why it matters to them.

How do others introduce you?

I think the last description of me was that I am the resident mad scientist. While I was still at Metabase, my time was split between data science and supporting engineering work. Trying to apply various machine learning approaches or heuristics to the products and experimenting on blue skies projects. My work revolves around using and making sense of data.

What are some misconceptions that people have around data?

One of my pet peeves is Big Data and the idea that the sheer quantity of data will ensure that you can get something relevant out of it. This might be the case when you really have a lot of data. To get there, you need a different approach. 

I often see people starting out and trying to apply some sort of machine learning or similar statistical approaches where they are hoping that throwing enough data at the problem will solve it. Still, they don’t actually have the quantity of data required. 

What I try to do is be smarter with both heuristics. Sometimes it’s better to have less data if that means you can understand it more clearly.

Example:

You want to analyze data, and your company is relatively at the beginning. Because of this, you don’t really know what kind of customers you have, but you almost certainly have a mix of different types of customers with different needs (customer segments). Those customers will behave differently in terms of how they use your product or how they behave during the purchase journey. 

If you add your data together and look at it, you will have quite a high level of noise because of those differences between the segments. It makes more sense to focus on one single segment and analyze that 1/5th of the data set. This way, data is much clearer. If you detect a pattern, it’s more likely to represent something meaningful for you rather than just being noise.

When analyzing large sets of data in a growing company, the thing to watch out for is that you don’t capture the different balances between the segments. As your company grows, the composition of your segments is going to change. This might mislead you to see trends while it’s just your early adopters being replaced by the early majority. If you instead focus on just one well-defined group, you can better see what’s going on.

Consider where the data comes from and the causes that generate the effects you are observing? Try to tailor the hypothesis about the processes you are observing through the data. This will make it easier to trace the problems through the system instead of focusing on the data layer.

This is also a way for people who try to break into data science or analytics to have the edge over their competition. It’s an advantage to have a good sense of the broader business context, the fundamental business metrics, marketing funnels, basics of UX, product development cycle, and understand the basic needs of users. This allows you to translate your analytics or data science problems into these business contexts. That makes you a better analyst than just being slightly more sophisticated with your statistical approaches.

What are some resources for people that want to learn more about this way of thinking about data?

This is my basic analytics knowledge stack:

What’s the story behind your offering of mock engineering job interviews to the local developer community?

I got the idea from being part of the SharpestMinds community – a startup that pairs people who want to break into data science with mentors from the industry. They practice doing mock interviews, and I’ve seen the value of this exercise to gain more experience.

I’ve also noticed that I’m one of the few people who don’t see hiring interviews as an absolute burden. I generally try to think about problems through processes, and the hiring process naturally allows for improvements. I’ve also had a lot of experience being on the other side – interviewing for different jobs. This allowed me to reflect on my interviewer style and start noticing patterns where candidates are weak.

Offering this to the local community seemed like a relatively high leverage way to give back to the community. So far, I’ve done 22 mock interviews: 18 developers (8 juniors, 7 mid, 3 senior) and 4 for marketing positions.

What patterns did you notice in your mock interviews?

The first significant pattern is that people don’t explain enough the intermediate steps or show how they think. Sometimes it’s okay, or I assume that you’re thinking the right thing. But you’re also not helping me understand how you think and how you solve problems. And this is one of the most important things that I need to understand. 

The goal of the interviewer

When I’m interviewing, I’m trying to answer three questions:

1. How’s it going to be to work with this person? I want to understand if I will enjoy working with you. If I don’t get that sense, I’m going to reject you as a candidate.

2. I’m trying to understand how high is your ceiling? What’s their potential? Most of the time, I don’t really care much about where you’re currently at as long as you fulfill the basic needs of the business. The process of finding someone, especially if we talk about developer jobs, is so long and so costly that it makes sense that you will invest in those people anyway. I’m trying to understand how far we can take these candidates with the mentoring.

3. What was their most significant project or task so far? What is the biggest responsibility I could entrust them with on my team? I find this a much better question to discuss instead of talking about seniority.

I notice that seniority tends to be subjective, and you also have these horrible proxies like 7 years of experience. It’s complete rubbish for what we’re trying to figure out. For me, it fundamentally boils down to this question – what would I be comfortable to entrust them with? This also tells me where they would slide into the organization, and it would also allow me to have a rough sketch of their career path. This will enable you to clearly communicate all of this to the candidate.

The second bad pattern is that candidates respond to the questions as one-way communication. We want you to engage with us in a discussion. We’ll sometimes give you intentionally underspecified tasks because we want to see how you act in a situation where you don’t have all the information at hand. If you try to directly solve it, you’ve already failed. When I give design tasks, I make sure they lack detail and concreteness, and I want to see where the candidate takes them.

There are more structural reasons why you should be asking your interviewer questions. By asking questions, you essentially take charge of the process and demonstrate how you would autonomously solve problems. If you just answer the questions, I don’t get a sense of organizing your thinking around different sections of the problem. The other reason is that interactive sessions are way more memorable and more fun for the interviewer. It also allows you to shift the interview into a shared problem-solving session and forces the interviewer to commit to trade-offs. Engaging the interviewer in the process will force them to pay attention to what you are doing. You can also show that you understand the connection to business and real-world needs when designing the IT system. We can also then talk about the needs of stakeholders and what kind of outputs you would provide for them. All of this will leave your interviewer with a better feeling of you as a candidate compared to a session where you would directly answer questions.

Where do whiteboard coding challenges fit into all of this?

I think whiteboard coding challenges are a poor use of time and inefficient. They suck both for the candidate and the interviewer. When I do the interviewing, I try to sidestep that. What I will do is I will have one or two questions, which are designed questions, but I want the candidate to walk me through their process of thinking and how they approach something.

Example problem

How would you design, at a web scale, a web crawler?

This question allows you to ask for further definition of the web scale. How often does it need to crawl, and what kind of data is it crawling? What’s the business case for this web crawler? It also allows us to discuss what kind of resources we have available and the trade-offs.

The same is often true even with algorithmic questions. These tasks are always embedded into a larger business context, and they often don’t have clean solutions. So you’ll need to be making trade-offs and thinking about how it’s going to be plugged into a broader architecture.

A similar example is to ask a candidate to design a data processing pipeline: we have real-time data coming in, and we want to do something with it. At this point, you can start by designing some sort of streaming pipeline, or you can start by asking business questions. By asking business questions, you might learn that we don’t need to access the data often enough, and you might be better off with an hourly batch process. This won’t demonstrate your knowledge of data processing. Still, it will show an extensive understanding of your role in the company. Also, the interviewer can always ask additional technical questions if they really want to know if you understand how to build a stream processing pipeline.

The more senior you become, the more it will be about solving business processes and less about the specific technical implementation.

How would your interview differ between a junior and a more experienced candidate?

I would still expect that junior candidates would have at least a rough handle on the trade-offs. Obviously in less detail than someone with more experience. I would raise the bar much more for more senior candidates and expect more detailed reasoning around inflection points. I’m also trying to see if you understand a broader context of what we’re doing. So if you’re interviewing at a product-focused web company, I would expect things beyond your role. What are some fundamental elements of UX, how product management works, what metrics this type of company would look at and why? This allows me to talk to you about things beyond just basic web frameworks and databases.

I’ve noticed in candidates at all seniority levels that people don’t really apply their knowledge. An example of that would be HTTP protocol which we all think that we know. Still, when candidates start talking about using it in practice, they tend to forget things like the payload limitations of the GET request. They must show that they have a deeper understanding of such details.

Example question

How would you design an URL shortener? What are some trade-offs and constraints that you’d need to think about and when?

Is there anything else that candidates should understand about interviewers’ questions?

Fundamentally a big problem when interviewing is how to disentangle skill and knowledge. An interview is inherently conducted in the medium of knowledge and not on a skill level. But as an interviewer, I want to know more about your skill than your knowledge. So when I ask a question, it is really a proxy to understand your skill.

If you answer my question in a school-like form of a definition, it doesn’t help me understand how you think and solve problems. Instead, try to talk about trade-offs and weave in your previous work experience. It shows how you structure your answers and how you prioritize the issues. Think of an inverted pyramid from journalism where the first sentence is the most important and tells most of the story. Then the rest follows with the least important information at the end.

Once you get a complex question, you can also take a few minutes and sketch out roughly the outline of the answer. It will help you keep a better structure, and it’s less likely that you’ll forget essential insights. If the interviewer refuses to allow you this, seriously consider if you want to work there. It shows a complete lack of humanness and introspection on how to conduct an interview.

Overall I find that job ads are very bad at describing the needs of the company. It’s better to just apply even if you don’t match the description. This is especially a problem for women who tend to see all requirements as hard requirements. What’s more likely is that once you get an interview, the company will try to figure out if you can learn to do the role and be trained fast enough to do the job required.

How can we improve our thinking patterns?

When you are a junior, one of the most critical skills almost nobody teaches us is how to ask good questions. What’s important is to learn how to make this transition between unknown unknowns to known unknowns, where you can start asking questions.

There is value in going in multiple directions in your learning and getting to know different concepts. For instance, you can always decide to read on the Annual Recurring Revenue Funnel framework basics. It’s a straightforward concept. You can read about it in two hours and then mentally file it for later. Later, when you approach a new problem, you have all these frameworks stored, and you can start thinking about how I would translate this problem to one of the stored frameworks. This allows you to ask better-defined questions, and there’s probably a community around those frameworks that can help you. There’s an extra benefit that if you’re stuck in a job without mentorship, these avenues can provide you with a community that helps you level up. Still, you do need to ask good questions. Otherwise, you’re not going to get good-quality answers.

One thing I’m exploring at the moment is how to better approach problem decomposition. It’s turning out to be an immensely high leverage skill because that gives you a framework to ask questions, even when you don’t have a concrete framework to map the question to. Also if you can write about it well, or if you can teach someone then you probably understand the problem. 

This is tied closely to thinking in meta terms about how I work, the friction points of my tools, and how I even approach solving a specific problem. If I’m mentoring on the job, one of the main things we will be talking about in our 1-to-1 is this priceless kind of meta-work aspect. It’s also a helpful topic if you’re leading a team and talking to them about meta work. There will be friction points pop-up and things where everyone thinks that they’re losing time and resources. Thinking about these issues on a meta-level helps the team improve their work.

How would you onboard a new junior engineer into a company?

No matter the seniority, I think we just need to make onboarding more efficient. Each new employee should have two mentors. One is your long-term mentor, and you report to them throughout your career in their department. They should care about your skill development and how to become a better developer and human. And then, I would also advise having a second mentor who helps with onboarding in the first few months. They’ll help transfer the institutional knowledge, and you can be more relaxed with them as you don’t have to worry about looking dumb in front of them.

Have important parts written down. Create a Tao of Engineering, the principles and beliefs we have as an engineering organization, and how we do things. This document is also going to address how to approach trade-offs and what we value. Examples of values could be UX, reducing friction, or having a very stable application. Things will fail at some point, and it’s easier to look at them from a systems point of view and fix the process instead of assigning blame. When you have clashes with the team, you can blame the Tao for not providing sound guidance and focus on improving the document.

What’s your advice for companies going remote?

My very strongly held belief is that if you think that remote doesn’t work for you, it indicates a bigger problem. Same if you’ve seen a massive drop in productivity after moving to remote work. It’s not an issue of being remote, but rather that some other processes are broken. In-person communication managed to smooth over them enough that they weren’t as acute as they were when moving fully remote.

What’s your advice on mentorship?

Seek out mentors if you don’t have one assigned to you. If you are in an organization where you can enact that, make sure you have a practice of mentoring. This should be deliberate practice, also from the mentor’s perspective, not just something that you do. It must be something that you care about, invest the effort, and make yourself a better mentor. 

In general, everyone in the organization should have someone they see as their mentor and kind of helps them with the guidance of their work. Mentorship for me means meeting at least once every two weeks, and it’s not like a quarterly performance review. It has to be a regular activity, almost kind of like a ritual.

If your organization is not offering that – seek out mentors. If you ask 10 people you look up to, one of those will probably say yes. It’s also a perk that companies can offer if they don’t have a mentorship program yet. Use some of the funds to bring in external mentors and spread the knowledge around.

Something like coffee or free food is worth basically nothing to a developer. Having an established mentorship practice will make a massive difference. As a developer, you’re going to use this job as a stepping stone to the next one, and you want it to provide you with this kind of value. It’s going to make you better at your job and also increase your future earnings.

What are some resources that you recommend for leveling up?

Developer oriented

Explore things outside of your direct domain of work and think about how you could integrate this knowledge with the rest of what you know.

What I learned from talking with Simon

Thinking about how we think and how we communicate is valuable on its own. I should have these meta discussions more often.

Interviews are just a proxy method to figure out if you’re a good fit. At more senior levels, they’re equally also about the organization that you’re interviewing with.

There are great frameworks (e.g., mock interviews, premortems) that can make outcomes less stressful.

— Meaningful work