you should really watch some of these videos on youtube, there’s quite a few of them and yes you’re 100% spot on that if I recorded myself doing this stuff I would be fired but some of these kids go into great detail as to what issues Amazon is having, the details of said issues, and potential work-arounds/fixes their seniors suggest to them. When on call only the new hire is paged and it’s up to them to page a senior or someone else on the team when they’re stuck. The problem is these kids don’t want to admit they’re stuck to their seniors or other team members because they feel it’ll impact them negatively. They admit to it. So I’d say 8 times out of 10 the tickets they get paged for don’t get resolved and are passed on to another team in the morning. So the whole thing is pointless.
In one video I watched they do have a shadow but it was reversed. the senior is the shadow and ONLY during the day. the new grad hire is still doing all the work. and after office hours they’re no their own. I wish I could find the video again as it was awhile ago but one kid recorded himself working on a ticket at like 3am and I was almost screaming at the screen like “NO DON’T DO THAT OH MY GOD PLEASE CALL SOMEONE!”
It’s like when AWS went down a few weeks ago I was thinking “probably one of these new hires at 2am trying to fix something”
I just really hope that Amazon at least has it set up so that the really important stuff goes to actual, trained SREs. They could set it up so there are queues for things that aren’t business critical and have a very loose SLO that get assigned to the new grads. Or, the new grads get paged when the error rate for the service is 1% and if it gets above 3% someone who knows what they’re doing is woken up. If say all issues with Amazon’s Route 53 DNS service is shunted to new hires, AWS would be going down constantly.
you should really watch some of these videos on youtube, there’s quite a few of them and yes you’re 100% spot on that if I recorded myself doing this stuff I would be fired but some of these kids go into great detail as to what issues Amazon is having, the details of said issues, and potential work-arounds/fixes their seniors suggest to them. When on call only the new hire is paged and it’s up to them to page a senior or someone else on the team when they’re stuck. The problem is these kids don’t want to admit they’re stuck to their seniors or other team members because they feel it’ll impact them negatively. They admit to it. So I’d say 8 times out of 10 the tickets they get paged for don’t get resolved and are passed on to another team in the morning. So the whole thing is pointless.
In one video I watched they do have a shadow but it was reversed. the senior is the shadow and ONLY during the day. the new grad hire is still doing all the work. and after office hours they’re no their own. I wish I could find the video again as it was awhile ago but one kid recorded himself working on a ticket at like 3am and I was almost screaming at the screen like “NO DON’T DO THAT OH MY GOD PLEASE CALL SOMEONE!”
It’s like when AWS went down a few weeks ago I was thinking “probably one of these new hires at 2am trying to fix something”
I just really hope that Amazon at least has it set up so that the really important stuff goes to actual, trained SREs. They could set it up so there are queues for things that aren’t business critical and have a very loose SLO that get assigned to the new grads. Or, the new grads get paged when the error rate for the service is 1% and if it gets above 3% someone who knows what they’re doing is woken up. If say all issues with Amazon’s Route 53 DNS service is shunted to new hires, AWS would be going down constantly.