[THEME MUSIC] [NON-ENGLISH SINGING]
MP: Hello, and welcome to episode 2 of season 2 of the "Google SRE Podcast"-- or, as we affectionately refer to it, the "Prodcast." I'm your host, MP. And here with me, again, as co-host is Pam.
PAM: Hi, everyone. Thanks for having me back.
MP: Glad to have you. Today, we are going to continue our conversation with earlier career SREs. And we have two more folks with us who have had a different track into SRE than our previous guests. And we're going to get to hear a little bit about their experience at Google and before Google today. So we're going to go ahead and let them introduce themselves.
MARIUXI: Thank you, MP. My name is Mariuxi Vasconez. I am an SRE for the corporate [INAUDIBLE] team, which supports the core infrastructure here at Google. I've been at Google for about six years, and I've been an SRE close to two years.
JULIAN: And I'm Julian Alarcon. I am a Chrome SRE on the Chrome team, working on the world's most popular browser. And I've been at Google for around three years and SRE at Google around a year.
PAM: Welcome, Julian and Mari. So I know that you two share a common thread in how you got started here at Google. We have what used to be known as ITRP, now called ITCP program-- the IT Career Programs-- and that provides equitable, sustainable, and well-supported paths from operations to other in-demand, tech-plus Google roles. So would the two of you like to share? Julian, why don't you start how you began your path into SRE?
JULIAN: Thank you, Pam. Basically, it was a weird path. Early on, I loved to break things, just via the computers. My dad, actually, had a little home office thing going on. And he would just have computers that were pretty cheap and I could break. So I used to work on them a lot. And moving into high school, I saw an opportunity within this program called Cisco Academy, which was working on networking and routers and switches. And a teacher really pushed me to take a chance and join the program that she was actually leading.
And through that, I just loved networking. And I learned that I want to do something with architecting, or building systems, or working on large-scale systems in any type of way. I didn't understand what it meant. It just sounded cool, and it was one of the main jobs in this poster that we had set up in that Cisco Academy room. Then moving into college, I took a different route. Instead of computer engineering and computer science, I went to IT thinking that, hey, maybe I can create my own path with what I wanted to do I started with IT. I started getting involved in school with actual computer science stuff and all that.
But then towards junior year, trying to figure out, you know, what do I want to do? I went back to that thought I had about being an architect. And I started looking up, OK, what is an architect in tech? And I heard about this thing called DevOps, which was connecting engineers and the operations. And then I saw this thing called SRE, which sounded like DevOps, but it sounded a bit more cooler because it seemed like it came from Google. It was a philosophy of how to deal with production. Didn't really understand production.
Then after a couple of classes, I realized in my IT degree, like, hey, production sounds cool. I want to be owner of production. And yeah, through that and my involvement with school through this honors society, I was able to get the opportunity to apply to the ITCP program. And just took my chance it-- like, hey, let's go to California. Let me go to a new state, meet new people, and see if I end up a SRE somehow. I didn't know there's an avenue to get into SRE. Yeah, I just took a chance.
And I also worked as a systems engineer for six months in Kansas City. I just wanted to get out and just experience what it is to work in the industry. I did have an internship within the school, within the university, and within my high school, but they were all IT related and never had really touched production. But yeah, I took my chance. Spend six months working on production. I broke a hospital's imaging system-- the entire hospital's imaging system. I felt really bad about it. But it did help me moving into Google and knowing that, hey, I can break bigger things.
PAM: Breaking things is fun, isn't it? It helps you learn. How did that lead into Google for you?
JULIAN: I took a chance on, hey, I'm having this job for six months, and I'm really enjoying it. But Google-- this is an opportunity you usually don't get. And the opportunity to work at Google actually came as a surprise to me, just because I never thought I would get the job. Because I never thought, hey, Google was never in my mind. Even though when I was young, I loved Google. I loved the products. My dad-- big user. Big, big user and big fan. So I applied for the role. Somebody reached out to me and said, hey, just send your resume in. Don't worry about it. Just send your resume in.
So I worked on it and polished it. My first interview date, I actually forgot about it. And I was at the supermarket. And they called me-- hey, are you ready for the interview? And my heart sank because I was not ready. I was in the produce section looking at bananas. Yeah, I rescheduled that interview, got my GCP shirt on for the next interview, and just kind of explained my love for computers and my passion and that, hey, I like breaking things. I like learning. And I don't know-- I want to take the opportunity to work at Google and see what avenue of work there is for me. I can explore my skills.
And yeah, somehow got the job. Still didn't understand it-- got a welcome package, like, hey, welcome to Google. Didn't believe it. I was like, OK, this is they're tricking me, you know. I'm going to get to California. And they're going to be like, hey, yeah, we sent that to the wrong person-- apologize. But yeah, got to Google after the six months at this other company. Started working on the ITRP program. There, I found out that there actually was an avenue to get to SRE.
And the program actually allowed me to do some work with SRE if I wanted to. And seeing as my last role I worked in production, and I did break smaller stuff-- I heard SREs broke bigger stuff-- so it just made a smart transition. Hey, let me go to SRE just because it is what I want to do. It's in DevOps-- that type of operations and computer engineering. And yeah, I took a chance and started working on my way to be an SRE.
MARIUXI: I started at Google as an IT intern back in 2015. And the natural transition for IT interns was the ITRP program for most recent graduates. So my experience in the ITRP program is that we were able to not only be exposed to the broader teams in Google, but also start learning more of troubleshooting skills, and learning the Google internal systems, et cetera.
So I think already there, I was already getting a little bit of IT SRE experience. Within the program at the time, we were able to also do our rotations. And there were a lot of teams-- SRE teams-- that used to recruit from the program. But at the moment, I wasn't familiar with SRE very much. I had some stereotypes that were talked about, about SRE.
So I was not looking into SRE at the moment. I created my own rotation at the time. I was interested on developing. During ITRP, I was able to also do fullstack certification. So I wanted to make sure that I was applying those skills that I was learning and paying for. So I made my rotation and went to work with a cloud team that was developing tools that help sales to keep track of their accounts and projects that they were working on with clients, et cetera.
My stereotype at the time for a SRE was that they were full-on software engineers. I was very intimidated about that thought because I did not see myself sitting and creating an entire database, very efficient, that supports millions of users. And the stereotype was also that mostly were men in that role with years of experience and expertise. And I was very early in my career, so I did not have that experience.
So I was kind of running away from it, but I was able to network with the right people here at Google. I was able to get to know a couple of SREs and have conversations of what they looked like. And getting to know the role-- getting to know the teams that are here in Google-- helped me to understand better and feel actually integrated and embraced, I would say, by the SRE org.
These are people that love to share what they know-- experts that will sit down with you and would explain your service, and how they made it more efficient, and how it is reliable. So I really like that knowledge-sharing. And I would say that was one of the first things that caught my eye from SRE.
PAM: You mentioned already you went more into customer engineering out of the IT program because you had stereotypes that maybe SRE wasn't going to be a good fit.
MARIUXI: Or I can continue if you want.
PAM: Yeah, let's continue with your story.
MARIUXI: So from ITRP going into my full-time role out of the rotational program, I landed a job with a customer engineer with Cloud. So there, we were helping small/medium businesses transition from whichever was their infrastructure, whether it was on premise or in another cloud provider, into Google Cloud. This helped me to better understand how startups or other medium businesses really planned their infrastructure and help them do that transition into GCP.
These already gave me a little bit of perspective into something that a lot of people talk in SRE, but it sounds like a very challenging thing to do, which is NALSD, the non-abstract large system design-- one of the biggest interviews, also from SRE So I believe that I had an advantage in that area because I was seeing so many companies, and I was helping them to translate whatever they had out there into GCP, and make sure that it's efficient, and make sure that it can support x amount of users that they have.
So when I was talking with other SRE colleagues that I had, we were exchanging these thoughts, et cetera. And at the moment, I felt like my role was more consultant because I was not able to touch the technology for the other companies because they were not part of Google yet. So there were a lot of liability risks there. So I felt like as early as I was in my career, I wanted to get more expertise with actually touching technologies, et cetera.
So I was able to create a small, 20% project time with an SRE team and work in a little project that was coding. But I was able to understand how SREs work on it. It's not actually, you know, the signing, creating services, but more like automating or, like a lot of SREs say, automating ourselves out of the job. So I was able to sit down on different meetings that really gave me perspective.
But I would say my way into a SRE was actually starting with that 20% networking with other SREs. And that's what made me want to be part and move into a SRE later on.
MP: It's really kind of funny how it sounds like Julian ran towards SRE, and you did everything you could to run from SRE. But ultimately, you both still landed as SREs.
MARIUXI: Exactly, yes.
MP: I'm curious-- this question's for both of you. What was the biggest challenge? We can start with Mari, and then I'd like to hear from Julian as well. What was the biggest challenge in moving from your previous role into SRE?
MARIUXI: Yes. For me, it was definitely the technology side of things, the coding interviews, and, of course, NALSD as well because thinking or talking about businesses that have already their infrastructure created and just moving into another provider is a little bit easy, in my words, to afterwards when I experienced NALSD.
Because I had to prepare-- and I don't even know how many mock interviews I did at the time-- to really understand the depth of a system and how there is engineers that sit down and calculate, you know, how many servers we're going to need to storage x amount of pictures or whatever it is. And then not only just thinking about that first idea, but, OK, how are we going to design the cache for this? How are we going to make this reliable if something goes down?
Like, there is so much depth into it. And I met engineers that have been years within SRE, and they still don't consider themselves experts. So I think that's something challenging-- really understand the depth of the systems. And I would say the next thing for me was learning more about the technology because, unlike Julian, I graduated in information systems.
And as a first generation going to school, I didn't have the privilege to take really technical classes, data structures, and all these things that would help me with my coding skills. So taking the time to learn those technologies took a while as well. But I was able to do it, thankfully, alongside teammates that were experts and would sit with me and explain me and help me to practice. So I would say that took some time of preparation on my end within Google as well-- extra hours from work. [LAUGHS]
JULIAN: So the question is, what was the hardest challenge moving to SRE? How large an obstacle it is to become an SRE is massive. I mean, it's the amount of knowledge-- specifically NALSD, as Mari mentioned-- it's probably the biggest thing. Just system design in general. Most of the time, if you're an entry-level engineer-- if you're an entry-level person in tech-- you're not really familiar with an architecture of a system, right? And that was the hardest thing for me, just to understand what Spanner meant. What the intricacies of a large-scale distributed database-- just the idea of that was hard to do, right?
And the only reason that I became an SRE was because somebody reached out to me and said, hey, there's a couple of people who are one year ahead of you that are training to start SRE interviews. And they're doing mocks. Do you want to join this group of people? So I'm like, yeah, let me join this group people. I want to learn what they're doing-- how they're doing mocks. And this is just when I started a couple of months in, so I had no idea what this design even meant. And I just started going to mocks and learning how they tackled system design, how they did coding interviews, just like Mari had mentioned.
I had to spend my work from 9:00 PM to 5:00 PM and then from 5:00 PM to 10:00 PM, a whole year and a half, getting to that level where I felt ready to do mock interviews, let alone the actual interviews. But what we did is that group-- a little group of people-- we started talking about, hey, you're a year ahead of me. What are the things that challenged you most? And then we started just sharing some information. More people started joining. And we just started, every single week, interviewing each other, like mock interviews.
Even if we didn't know how to do it, we would just mock each other, and mock each other. Just like Mari mentioned, I think I did over 100 mock interviews. I just wanted to be an SRE so badly that whatever work I had to put in, I did. And it took a lot out of me. I can definitely say that my mental health is affected. It was during the pandemic, so also do mocks while I'm dealing with all these other obstacles. So it really was probably the hardest obstacle in my life, to be honest with you, to become an SRE. But I had good people supporting me and people who converted into those roles.
And I do want to throw this in, though. This group continued even after I converted. Then I helped others. And now, people in the same floor that I work with, two people are people that I mocked continuously and were able to convert from ITRP and ITCP over to SRE. It's so, so hard to get there, but I feel like a community helping each other, and quizzing each other, and challenging each other to be better can achieve anything.
PAM: It sounds like you and Mari both had a pretty strong support network and people that were helping, mentoring, and helping grow your career towards SRE. What was the most helpful experience of the mentorship you've received?
MARIUXI: I would say that the most helpful thing they shared with me, outside the technological parts-- their expertise and knowledge-- I would say they gave me confidence that it's OK not to be an expert on something. That it's OK to not know everything from a system because you have a team. That you can rely on your team. You're never going to be alone on call. You're never going to be blamed for any outages that you provoke.
It's so funny because the first time that I was on call, you know, I had double-checked everything that I had to do to help a team migrate from one of our data centers into another failover. And I had sit with the TL. I was like, OK, this is what we're going to do. Are these steps OK? He was like, yes, no worries. I'll be there. And I was in Dublin during this time, so he was in New York. I started at 7:00 AM. It was, like, 1:00 or 2:00 AM here in New York.
But I remember I went through the playbook because, of course, we have helpful documentation as well. And I started shutting down the different VMs because we were going to be failing them over. And then out of nowhere, I received a ping from, like, three people. And they're like, did you start the migration yet? And I was like, yes, I'm almost halfway through. And they're like, no, you needed to wait for us to give you a heads-up because we have to fully, gracefully shut down these machines, et cetera.
But this is not something that we didn't talk about with my TL. I guess that was not part of the playbook. So I quickly reached out. They were like, it's OK. For next time, this is what we have to do. Please update the documentation. Thankfully, nothing was lost. Everything was fine. But that was my first time being on call. And you know, even though it was like a rookie mistake, it was OK.
I felt comfortable talking to people. They were OK. They didn't get mad at me. My teammate reached out as well. And I stayed through the rest of the migration just to make sure that nothing goes wrong and I don't feel pressured. So I feel like it's a supportive organization. And just knowing that I'm not by myself is a good feeling. And the best thing or best advice-- that reassurance, I would say, that they gave me.
PAM: And a good learning opportunity as well, right? Because if you hadn't made the migration happen, no one would have noticed that there was something missing in this playbook to do that type of change safely. So never waste a good-- what's the saying, MP? Never waste a good outage?
MP: I don't think I know that one.
MARIUXI: We just made it then.
MP: Never waste a good opportunity for a postmortem?
PAM: I guess, yeah.
MP: I just like it that it's almost sort of a badge of honor as an SRE for how big the dollar value on the largest outage you've caused is. Because it happens. We can never be perfect. And we're always going to accidentally do the wrong thing here and there.
PAM: And with the scale of Google, there's always a chance that something in the intricate systems is going to break and go wrong. And what is great is, as Mari was saying, no one blames the person. It's not any one person's fault. There's just too much complications in the system. And our job, as in Mari's example, is trying to make it clearer-- updating those playbooks.
MP: Or even going so far as the next layer of, well, instead of having a human manually do that process, right, the tool that ensures you gracefully do everything and the human doesn't have to think about all of those intricacies. Yeah, but then your script's going to be out of date six months later. Then it breaks in a different way.
MARIUXI: It's just automating ourselves out of shutting down machines.
PAM: Exactly. So Julian, do you want to share a story about your experience with mentorship and getting into SRE?
JULIAN: I want to go back to when I first looked at what SRE was coming into ITRP/ITCP. So here I am saying, hey, I want be an SRE somehow. I don't know how, but I want to get there. And I knew that I needed somebody in SRE to help me get there because there's a lot of stuff that I don't know. And the program, like Mari mentioned, there's a lot of avenues you can go to. You can go be a custom engineer. You can even do SWE work. You can do what we call COE work, which is more of on the IT side.
So I went and reached out to every Latinx SRE at Google I could find. And I just started looking at people and connecting people who are in our HOLA, which is our Latinx interest group inside of Google, and just started seeing people in SRE. And I found a couple of people. I met with them, and I told them straight up, hey, I need a mentor. I want to be an SRE somehow. I don't really see people who look like me in SRE. So that's why I'm reaching out to you, because you make me feel comfortable. And you can ask questions that I usually wouldn't ask somebody who I don't feel comfortable with.
And one person within the Android SRE team went ahead and said, yeah, let's meet every single month. And it was such a-- it was a journey of meeting twice a month with this gentleman. His name is Guillermo Quezada. And just asking questions like had mentioned, what is locking mechanism? How do transactions get locked? How do systems work? How does Google work, you know? And it's a big question with a lot of answers, and it probably takes years to even understand, like, couple of things at Google work, and Google does, and how they do it.
So it was talking more about the theories, and what SRE thinks, and [INAUDIBLE] these postmortems. How to understand how SRE can help teams be better and create services that are more reliable. So all these things-- all these theories-- he just started tossing them at me and having me read things, research documents, design documents. And through that-- through him-- I really felt comfortable with what SRE was and telling myself that, yeah, this is what I want to do. This is pretty, pretty cool.
Again, I go back to breaking things. You can break bigger things. And yeah, and through him, I was able to eventually start mocking to the point where he gave me the mocks that he would ask interviewees outside of SRE, who would be more senior folks, and just quiz me at that level, you know, get me to the point. I had already been practicing in my group, so it's like you know you're at that level. So let me push you and just ask you next-level questions to see if you can come up with ideas with what we talked about. And because of his help, I can genuinely say that he gave me that reassurance and support to keep pushing forward and to ask interesting questions.
MP: So something that we've heard from both of you today that I'm wondering if we could maybe highlight a little bit more-- or maybe, where's the room for improvement? Because you've both mentioned these particular experiences of not seeing people that looked like yourself in SRE, and that being a bit of a barrier to entry for you. And you both overcame it, but what do you think? What would be kind of our little, if we were to do a quick two-minute mini postmortem, what is something that we could do better in those dimensions to make sure that SRE is welcoming to all people?
MARIUXI: I can start.
MP: Yeah, go ahead. I know this is not an easy question, and I'm not expecting a 30-second solution to any of this at all. But I think it would be great to hear from both of you about, maybe, where there's room for improvement.
MARIUXI: Definitely. And I speak not only from me, but I'm very passionate also about DEI. I think this is a great opportunity to help us show and create representation out there. I'm sure plenty of people can hear-- my accent is not fully English American. So yeah, this is a great outlet to start creating exposure for underrepresented groups at SRE. What's their story?
Because I think it's important to know, historically, that a lot of roles in tech have been fulfilled by privileged people that were able to come from households that had engineers-- had doctors-- that could guide them through these paths into SRE, into software engineering, whatever it could be. That's not the case of a lot of underrepresented groups-- a lot of immigrants that come to this country.
So there are programs like these. There are programs like ITRP. There are programs like Cloud Technical Residency or different residencies that Google offers-- not only Google, but other companies. It's just a matter of sharing and exposing that information. And I think that's not only the goal for Google, you know-- have information accessible for everyone.
But outlets like this one can also be shown to other potential SREs that want to join SRE at Google that may not come directly from university into SRE that are here in other roles. There is a way. Any technical role is just a matter of learning, practicing, and executing at the end.
So yeah, sharing these stories and letting people know that there are programs, and there are paths, and there is people that want to share that information and their knowledge with you is important. And little by little, we'll grow the representation at Google. The representation of what an SRE means-- who is an SRE-- and will hopefully bring in more people.
MP: Julian, do you have something you'd like to add on that thought?
JULIAN: I completely agree with Mari on everything. I did want to add, I think the biggest obstacle right now is just letting folks know that SRE isn't just your 20-year-plus experienced production engineer working on the backend service. People, they come from all walks of life. And for example, in campus events are events where you go and meet students or folks who might not be familiar with what an SRE looks like. And then, OK, take that image away and show them that, hey, we-- all types of people. Come to SRE with the goal of doing something completely different than the other.
But they all care about one thing, and that is making services reliable. SRE can also do a lot of things. So there's a lot of fun for anybody who is interested in tech and is techie and wants to, like, play with stuff and actually have impact on production systems. It's important to let people know that that's an avenue everybody is welcome in. So I've worked on this nonprofit which tries to incentivize minorities into tech. And we specifically focus on having workshops that teach students and folks who are wanting to get into the tech fields tech skills that might not be available through a school curriculum.
And one thing that we do focus on is having events where we have these really big tech companies like Twitter, Meta, Amazon, and Google just come represent and be like, hey, these are engineers. And we really focus on having people who look like the students and people who are just your everyday walk of life. Like, I don't want you to walk around and be, like, hey, that person looks like an engineer. First of all, that's making assumptions.
But second of all, you never know. You never know who that person, that you've never would have imagined works in SRE, is a lead engineer who-- they are leading one of the biggest orgs inside of Google or a big tech company. And they're having massive impact, making sure that every time you load up a web page, it loads in less than 5 to 3 milliseconds. Yeah, keeping that-- taking that image away, I think, is key.
PAM: Mari, when you landed in SRE? How did you go about finding a mentor and get comfortable in the role?
MARIUXI: Yeah, that's a good question because I did have mentors that were SREs, but they were not SREs in my team. And of course, as we were talking before, Google is very complex plenty of service, and there is plenty of people working in one service. So actually, for my team, we support different services. So it can get a bit complex.
But when I started, Google does this thing that is called SRE EDU. So all the SREs that start at Google have a week-long training on production systems, being on call, being on dashboards, and whatsoever. So during that time, I started with someone that started in my team as well. Of course, there is a difference between looking at production services and the core ones that we run, because they run completely adjacent from the production ones.
And so as part of being in the core virtualization world, we use our own servers. We work on the virtualization for the servers. And we provide the products to internal teams within Google. So one of the reasons why I went into this team to start with is because I wanted to learn virtualisation from scratch. I was working with Cloud. I understood what a compute engine, was what a VM is, et cetera.
But I never saw it and thought, you know, what workflows run that make these virtual machines, et cetera. So that's something that attracted me to this team. And so when I joined, I saw that we work with third-party technologies, like VMware, like NetUP to provide storage, [INAUDIBLE] which is an outsource technology that we have to create virtual machines and run them, et cetera.
But these are things that I heard about, never really seen. So actually, my teammate he had, like, 10 years of experience with working with VMware. And you know, we had to start getting ready and trained to be on call-- on duty for these technologies. And I was very new. I just passed interviews. I got into the role. So I really needed to learn more of, like, hey, what's VMware? How am I going to support this?
So I literally reached out. I'm very outspoken, in a way, or not shy in that regard. I like to ask questions. If I'm not familiar with something, like, let's talk about it. Let me learn of it. Let me read about it. So I sat down one day during a one-on-one that we had. And I said, hey-- and shout out to Marko Zagar, my teammate-- I was like, hey, Marko, I know you come from working with VMware for 10-plus years.
Can you please like have one-on-ones with me where we talk about VMware, where you can explain to me, maybe, concepts that I don't understand from the onboarding documents and playbooks that we have? So he happily sat down with me during a lot of months to talk to me and walk me through VMware-- also had experience with NetUP. Marko is just awesome. So any questions now that I have after two years, you know-- because I'm still learning and you'll never stop learning at Google or in any technology field-- I feel comfortable to just reach out to him and have him explain abstract things, or concepts that I really don't understand, or create some confusion to me.
So that's how I created that mentorship, which it wasn't like, hey, Marko, be my mentor. It was, hey, Marko you really know about this. Can you please help me understand this? And now it's just, yeah, a relationship that organically developed into a mentorship. I see him as my mentor and trust his judgment to give me advice in any project or any decision for the team. So that's how that developed.
PAM: I just wanted to highlight that Marko is from a different background than you, Mari. So your mentors don't have to necessarily look like you, as long as you're willing to reach out and find them.
MARIUXI: Definitely. Definitely. And I think even our personalities are very different. He helps me with a lot of technical stuff. And I'm happy to help with things that maybe he's not very comfortable with, like we're working on a project, so I'm happy to help more in the project management side of things, or tracking, or planning customer meetings, or gathering requirements that we need from the customers that we support.
So I'm happy to help out with those things and then also chime in with the technical part of things. So it's a good, balanced work relationship.
MP: So thank you, both, so much for your time today sharing your experiences and your insight. Pam, did you have anything you wanted to say before we closed out?
PAM: I don't know. Do I? [LAUGHS] Well, thank you both for joining. You've been great. And they're both such inspiring stories. And I'm glad we can share it with the world.
MARIUXI: Thank you so much. And I encourage anyone that's looking into SRE, or the name has ever ringed their ear, to just look more into it. Talk with people that are in SRE. Reach out-- it's a very inclusive organization. And I couldn't be more happy to be part of it. And I'm sad that I ran away from it for so long.
MP: Thank you so much. Take care.
VOICEOVER: "Prodcast," the Google SRE production podcast is hosted by MP English and produced and edited by Salim Virji. Engineering by Paul Guglielmo and Jordan Greenberg. Javi Beltran composed the musical theme. Special thanks to Steve McGhee and Pamela Vong.