Today we are joined by Jem Young, a senior software engineer over at Netflix.
Jem is here to discuss his programming philosophy and how it is an extension of himself,
how engineers should have the freedom to have ideas and veto things, the difference passion makes, and being informed about the libraries that you include in your code and if they are actually needed.
Netflix's homepage got 50% faster when Today Edwards had the idea of not shipping
side. Finding this balance of developer productivity and performance is important. Jem talks about how you should really examine the reasons why you are including a library in your codebase, and to
ask yourself if it could be implemented in a simpler way with what you already have.
Jem discusses what it means to love the work you do and to have a real passion for creating
software. Passion makes a difference in a developer's career path, the devs who are just looking to make
a paycheck and don't really care don't end up in the same place that those who really care about their work do.
Jem is curious, he experiments with software on his weekends, he'll sometimes wake up at 1am with
the solution to a bug he's been thinking about all day, software is an integral part of his life beyond it just being how he puts food on the table.
Trusting your engineers to know what they are doing
Weighing the performance costs of included libraries
The difference that real passion for software makes
How programming is like solving a puzzle
How the code you write represents you
John Lindquist: All right. Today's guest is Jem Young. So Jem, you want to go ahead and introduce yourself.
Jem Young: Yeah, thanks John. Like John said, my name is Jem Young. I am a senior software engineer at Netflix. And a fun fact about me, I am very tall. I am much taller than I look in all my pictures.
John Lindquist: You are.
Jem Young: And you're also tall, John. I don't think people know that either. When we stand next to each other, I don't feel uncomfortable because we're about the same height.
John Lindquist: We do. We've got a nice little eye level thing going on.
Jem Young: It's really nice. The Twitter avatar's don't do the height justice. We could use each other for a scale, but that wouldn't help at all because we're both tall. So maybe like, they're both average height guys.
John Lindquist: Exactly, exactly. So the topic of today I wanted to talk to you about is you work on a lot of sites that kind of avoid frameworks and avoid libraries due to performance being like the number one concern, right?
Jem Young: I wouldn't say a lot of sites. It is one of my mantras, and the team I work on with Netflix is something we believe in.
John Lindquist: Right. Yeah. So performance above all, first paint and all that, above all. So what do you see when on your projects that are like that, what are the requirements of a typical site like that? What requirements come your way, like, "Okay, this, we need to avoid all the extras, all the fluff"?
Jem Young: To back up a bit, we have the benefit of being able to AB test something. So we just don't roll things out and say like, "Oh, I hope this is work. I hope this gets the right metrics." We can AB test it. So I can say, "Hey, I want to use ... I want to build one experience with this library or a slightly bigger payload or something like that. And I want to build another experience with a smaller payload and no library." And we can see which is actually the better experience in terms of metrics and in terms of any sort of like user conversion. Or any other real world metrics that we care about. But generally, thankfully, my teams pretty well-informed about adding ons bufferless libraries. It's so tempting to just yarn and install, our MPM install, or yarn add, should I say, some libraries, that solves your problem. But, I always preach this, is just being cautious about what you're adding in. And do you actually need it? Do you understand what the library's doing? Can you do without the library? I don't know. I think that's something that everybody should keep in mind these days.
John Lindquist: Right, right. So do you find on the projects you have, are you ... How often are you actually able to remove a library? Do you have some sort of benchmark or quota or whatever that comes your way, where you're like, "I need to get this much faster or this much smaller"? Or you have to start finding things and removing and writing it yourself?
Jem Young: Thankfully, we have real time performance metrics. So it'll tell us thing like time to render. And I can break that out into different devices. So I say, "Oh, this site on my $3,000 MAC book, it renders in under a second. But on this low-end Android phone, it takes 10 seconds. That's unacceptable. So, at that point, I take a look at what is causing the lag. Is it something we can fix. Is it a product of our ... the experience we built. Almost generally, it's due to usually a slow API requests or just the size of the libraries too large. Or oftentimes, it's just something blocking rendering. But you would be surprised how often it's just bad measurement as well. That's a totally different episode we could do. But yeah, knowing that your measurements are accurate is kind of the first step in understanding, should I remove the library, should I not? Because there is a cost to developer productivity by say, I don't know, taking out React or something like that. It's easy for you and I to say, like, "Oh, just use less libraries. But in reality we're developers. We get paid to build things within a certain timeframe, and we can't just remove everything. So taking a more holistic view of your application and kind of the bottlenecks and whether or not you need to remove a library, something that has to be done on a case-by-case basis.
John Lindquist: Right. You do need to ship something. You can't just ship code that will do that something in the future. It needs to actually deliver a product.
Jem Young: Exactly.
John Lindquist: Yeah. I've heard you say a few times a code that's never written never has bugs. So I mean, it sounds like you have the power to say no a lot. Do you find that's the case that ... Throughout my career, I've been been on the receiving end of just a stack of, well, I guess PDF, so not a stack of paper, but a stack of PDFs, whatever you call a group of PDFs, and inaudible 00:07:46 implement this. And I didn't have a lot of freedom to say no because I was a consultant or I was an employee, and I was on there getting paid to do what they told me to do. I had a little bit of leeway in I can't do that before next Friday. This is gonna take six months, not two weeks after you deliver the proof concept, and then they think that you're done. You've been there before. But it sounds like you've been emboldened, I guess, with the power to say no. Do you find that to be true or-
Jem Young: Yeah. That's absolutely true, and I-
John Lindquist: It's awesome.
Jem Young: I have else been on that side where it's kind of unilateral. It comes down. Product is looked at, designers looked at it, all the relevant people have looked on it. And as the engineer, you're kind of the last wrong in the ladder that it falls down to, they're like, "Hey, build this thing." And I feel like in a lot of organization, that's how it is. But no, I'm absolutely emboldened to say no. I disagree with people often. In fact, that's not one of our core values, but it is value to disagree with somebody, have an opinion but don't be opinionated because I think one of the worst things for engineer or one of the worst things for senior engineer is to not have an opinion on things because you've made it this career. You definitely need that hard learned.
John Lindquist: Preach it. Yeah.
Jem Young: Well, sorry.
John Lindquist: I'm saying, preach it. Keep it going. Hallelujah.
Jem Young: There are expensive lessons that you and I have learned, John, once that we probably won't care to pick because they're so embarrassing. But I've broken production before, not Netflix.
John Lindquist: Through rite of passage.
Jem Young: It is. What do they call it, the ... It's called the 23rd hour mistake or something like that. Because your senior enough to not make a rookie mistake. So when you screw up, it's big time.
John Lindquist: It's inaudible 00:09:51.
Jem Young: Like the inaudible 00:09:54 database or something like that, which I've never done either. But I've worked in production. I've made a lot of bugs.
John Lindquist: I mean, we haven't the Western incoming missiles to Hawaii like that one. I haven't done that level, so I feel okay with myself.
Jem Young: I felt bad for whoever did that because at software engineer, there's a lot of unintended consequences that we just don't know, don't align. And it's impossible to think through all of them. And sometimes it's just bad luck, sometimes just bad design. But-
John Lindquist: I try to feel sympathy or empathy for pretty much every website or piece of software or video game or anything. Right now, there's some developer, maybe not the most experienced in their role trying to get by, trying to learn on the job, trying to ship through requirements, and then people complain about the bugs, and they tear it apart, and they rip everything, and it was their best effort. They put their heart and soul into it, and they did their best, and then you got online, and people just tear into pieces.
Jem Young: Oh, come on. Yeah.
John Lindquist: I hope they tried. I'm pretty sure they tried.
Jem Young: So I will say this. Not everybody cares the same amount that you and I do. There are lots of lazy developers there or developers that just try to get the thing done and then walk away, whereas ... You and I have spent time together. So I can say confidently if there's a bug or some lingering thing, we're not gonna go home and be like, "Oh," just turn on TV and hang out. We're probably gonna go fix it because it bothers us at a deep level that our code represents who we are. And I'm not just gonna this outstanding bug that I know about. But not everybody is like that. Not everybody thinks that way. I would love everybody to think that way. But it's just the way the world ... I mean, I'm ...
John Lindquist: I'm sure that's true across all career fields. I've had people come out to me just throughout my life and say, "Hey, I'm looking for a job. Do you know of any?" I'm like, "Well, do you have a particular interest?" Like, "Well, I'm interested in making money." Like, "Well, that's not a good start. Weird choice." Because I think some people are just in it for the paycheck. And whatever career you're in, if that's what you're in it for, then you're not gonna be happy, and you're not gonna find that joy in it, so.
Jem Young: Preach it, John. Preach it. We can do another episode on these in ... This is gonna sound mean, but it's not against coding boot camps. There are a lot of good ones out there. And there are people that work in Netflix that graduate from coding boot camp, don't have CS degrees. That's perfectly fine. I think, and this is definitely my personal opinion, there are boot camps out there or schools or for-profit universities that say like, "Hey, become a software developer in under whatever time, and you'll get paid a six-figure paycheck or $80,000, or whatever it is. And people were like, "Oh, yeah. It's kind of a get-rick quick scheme." But like you said, without the passion, you can be a software engineer, but you're never gonna be good at it because it's those little things, those nagging bugs that kind of like haunt you all weekend. And like, "No, I'm not gonna relax until I fix this." Or, "Oh, 1:00 AM." Sometimes you wake up out of bed and say, "I figured it out. I know it's a inaudible 00:13:31 problem," then you go code it out. That is one of the differentiators between a software engineer and a great software engineer. Not saying you need to kill yourself working overtime. But I see it in some people, and I don't see it in others. And years later, it tends to play out exactly how I think it plays out. Those people that push a little harder, that have that passion, tend to rise to the top.
John Lindquist: Yeah. I think what's fascinating to me is that's pretty evident from early on in the learning process. The first project they pick up, whether they're trying to make a circle move across the screen or trying to make a button click, submit a form or whatever, they are in it to figure it out. And they will not let anything get in their way. Obviously, a lot of it is the brute force approach, where you try every single option, and you know that every failure is just a stepping in the right direction to finding the solution, and you're just pounding a way at it. And there's just self-taught programmers out there who have learned that way, and they're wonderful programmers now because they are not afraid to fail, and they want to do it.
Jem Young: Exactly, John. Exactly. You could argue there are no failures when you're learning because you learn something. And to circle way back sort of earlier, point is that's the level of people that I work with. So I don't have to say, "Hey, you build this feature, and you added the library. There's an extra 2 megabytes or something like that." I don't have to worry about that because the people that I work with are thoughtful, and they know better. And that goes back to if someone asked me to build a project, I can say no because I have years of experience. And not like I have user experience working at a company, but experience in those after-hours learnings and mistakes that I've made and talks that I've watched and classes that I've taken and just random, boring, silly things that I've build that have no purpose whatsoever rather than just me building something. From all that, I have all that experience. So when I say no, it means something. It's not just I'm lazy or I don't wanna build it. There's no I'd say no for a reason.
John Lindquist: Yeah. I've tried that. I know that that's not gonna work because I tried it on some stupid project on a weekend one day when I was interested in looking, yeah, at a blog post, and I tried it, okay. You're contrasted me that it's not gonna work.
Jem Young: Exactly. I always say, for organizations that are not like that, that they say, "Well, you know what, John, I disagree. I think you can build it. Go ahead and try." And that happens quite a bit. From just talking to people around the world, that's kind of how software engineering is done in a lot of places. So those people, I say, like, "Well, why did you hire me then if you know better than I do? You hired some of the best in the world, right?" And they say, "Yeah." Well, why not trust them to make their own judgements on these things like this? My fiancee, she's a data scientist. She could run circles with me on SQL queries or just complex math that I can't even begin to understand. So it's not my place to be like, "Oh, you're doing that wrong." That wouldn't make sense. And so same way if someone's like, "Yeah, that's a good opinion, Jem. But I think you should go and build it anyways despite your reservations." To me, it's the same principle. And I don't understand that way of thinking because why did you hire these people if you're just gonna not listen to what they say or give them the freedom and latitude to build how they wanna build within reason, obviously.
John Lindquist: Yeah. One of my notes from one of your slides is believing people, not ideas. I think that kind of ring is true is that the people are the source of the ideas, where they're the ones with the experience, they're the ones who've come up with the possible solutions, and you have to trust when you put the people first that their energy and their focus is in the right spot, and the idea is they come up with the proper support behind are the right ones even if it's the idea you don't agree with. I don't know if all managers have learned that lesson yet. But people are like idea machines or solution machines.
Jem Young: Yeah. Exactly. There's nothing more powerful than a passionate software engineer. I've had the privilege of working with a lot of smart people over the years. And I just see the look in their eye when they're like, "Oh, I had this idea." They may come back a week. It could a month. It could be a few months later. I built this thing. And a lot of times, it's just revolutionary. Sometimes it's silly. But they learn something. But that passion is kind of what pushes us forward. And it keeps us ... Companies that empower those people tend to stay ahead of other companies. They tend to be the leaders and not the followers because ... It's like if you had the chance to, I don't know, hire a great artist. You had the chance to hire inaudible 00:18:50 or something back in the day. You're not gonna tell him how to create something. And you just have to kind of sit back and let him do his thing, and brilliance will come out of it. And it's the same way between ... Actually, I'll put this to you. Would you say software engineering is more of an art, or is it more of a science?
John Lindquist: Yeah. So I've thought about that one a lot kind of over the years. I think it falls under creative problem-solving that every problem we're presented with is a puzzle. And I think that software is an art inasmuch as the final, a puzzle, once you put all the pieces together is a piece of art. So my wife is actually amazing at puzzles. So it's one of those things that come up to my brain. So yeah. It's like the puzzle, once assembled and put up on the wall is art. But all the pieces lying there on the table before it's created, you're not necessarily painting art. You're assembling art. That may be a bit too out there to make any sense.
Jem Young: No. The puzzle analogy is perfect because that's a lot of what we do is we put together pieces. Sometimes we have to write the glue. But individually, the pieces are nothing. They're like, "Oh, yeah, that's pretty puzzle piece of flower," something like that, cool. But I wanna tell it doesn't mean anything. But we put it all together. So yeah, I agree. I think creative problem-solving is a good way of putting it.
John Lindquist: Well, I also think ... I mean, we spend so much time with interfaces and APIs and putting pieces together that once you've finally assembled your final project, you can still have a feeling of, "Ah, something is not right in the way things connect together, like with the way we're making our backend request." It's not art yet. There's some sort of imperfection in there that you can see as a ... From your experience, you can see the imperfection in the system. And they're like, "Well, we got to ship it." I know it's not beautiful. It's a product. It's not perfect, but it's out there, and then you can refine it and turn it into more of a nice beautiful piece of art later on.
Jem Young: No. Exactly. The code you write represents you. It is a part of you. I know we're going really philosophical at this. And some people, we've probably already lost crosstalk 00:21:29-
John Lindquist: Our podcast, we talk inaudible 00:21:31-
Jem Young: Yeah. What's the point in having your own podcast if you can't do whatever you want. But no, I always say I wanna be righteous in my code. And not at any sort of religious connotation, but just in a ... If I did something, there is a reason why I did that way. So anybody can go to any line of code that I wrote and say like, "Jem, why did you do this? This doesn't make sense." Or, "I'm confused." I'll be like, "Oh, I did it because this, this, and this. Because I wanna be thoughtful in my approach." And even if I'm wrong, I wanna have a good reason behind why I did it, not just, "Oh, I happen to do it." Because every line of code I write to a large degree represents who I am, just like an artist would say that painting or that that drawing or that statue is a part of who they are or a poet is sharing part of kind of their soul, their experience. And I know, it seems kind of strange to think of programing in that sort of sense. And that's definitely not in my head every day when I'm writing code. But to a large degree, I mean, we're still focused on individual lines that we lose the big picture quite often.
John Lindquist: Exactly.
Jem Young: Yeah. But I agree with you. I think software can be beautiful. I think systems can be beautiful. Designing a big complex system, that just is doing all these things, and it's just few minutes can't even comprehend the level of data complexity is going into it. No. I think that's beautiful. I know it sounds silly, but that's the way I think of good software.
John Lindquist: I agree. I have to share one story about trusting people, and I guess don't stop me if you've heard it because well, people might wanna hear it. So it's either UPS or some big delivery system where they have conveyor belts. And at the end of conveyor belts, it the boxes were too light, it meant they're packed improperly, and they shouldn't be loaded onto the van. And it was causing a huge loss in revenue and sales or whatever because they were putting incorrect boxes onto the tracks and going out for deliveries. And so they invested this huge effort into researching new ways of building conveyor belts so they could weigh the boxes and put the appropriate ones on the track. So they spent millions and millions of dollars on inventing a new conveyor belt. And so the people who were loading the boxes are under the track. They installed the new conveyor belt, and they're loading the boxes under the track, and everything is just fine. And one day, the new million dollar weight system break stone but they realized that the people working at the end of conveyor belt who dealt with the system every single day realized what they were trying to do. And so all they did was they kept things going by putting a giant fan at the end of the conveyor belt to blow off the light boxes so that only the heavier ones made it onto the tracks. And so when the people, like the senior managers and the CEO, when everyone came down to investigate what was going on, they saw a giant $20 fan solved the same problem that they invested million of dollars into solving. And I think that's that wonderful story of, you know what, there are people who deal with the thing, and you don't try and solve that problem for them if they understand the problem better than you do, anyway.
Jem Young: No.
John Lindquist: I'm sure I got a lot of details wrong in that. But hopefully, the moral came across.
Jem Young: No. That's a great story. That's a great analogy because that's so true. That's 100% true.
John Lindquist: I like that.
John Lindquist: Bring it to full circle. Let's hear it.
Jem Young: It's easy to like, "Oh, this noted library out there came up with this package, and it'll solve all of our problems." And we just installed, and we spend most time in integration and things like that when the actual solution could have been much simpler if you put a bit more effort into it, and you ask the people who are directly involved, "Hey, is there a better solution?" I forget who said it. But the famous saying that perfection isn't achieved when there's nothing left to add but when there's nothing left to take away. And I feel the same way with software is when it's down to its core, it's bare minimum that it needs, but also extensible. So it's not just highly coupled and does this one thing well. But nothing else needs to be taken away. That's where I'm happy. Because if something goes wrong, I understand the entire system because I'm like, "Okay, there's not much to it." It's really simple that the less code there is, the less chances of bug. It's simple, but no one ever thinks of it in that terms.
John Lindquist: I like. I think we'll end the podcast right now.
Jem Young: Okay. It's a lot of philosophy.
John Lindquist: That was great. I loved it. All right. I'm gonna stop recording.
Jem Young: Okay.
John Lindquist: See you, Jem.
Jem Young: See you, John.