My last post was from before the world exploded. The current events related to COVID-19 have made it tough to write technical posts, so I'm going to mix it up a bit and talk about something else: the chasm of self-doubt.

This post was inspired by a conversation I had with my team the other morning. We were troubleshooting a partition-based GC process and after looking at it a while folks started doubting what they knew about partitioning. This is common in our line of work. Things in the SQL Server world change quickly, and it's not uncommon for old myths to be taken as truths (like table variables being stored completely in memory). I think for an outsider, this amount of self-doubt and questioning might look unproductive and unhealthy, but I have a different take:

Ideally, all engineers of any kind should always have their legs dangling into the chasm, at a minimum. It's the only thing that protects production.

falling in

Falling into the chasm can feel like an uncontrollable downward spiral. You will find yourself researching things you thought you knew, and often finding that you were right. Sometimes though you do discover that you weren't 100% correct about something, but that small misunderstanding hasn't had much impact on your work. Other times you may find that you were entirely wrong about something. This can be a bit crushing, but the important thing is to acknowledge what you find and learn from it. Nobody can get everything right all the time, and as long as you learn along the way, things will be fine.

dangling your feet

Back to my quote above, I think self-doubt is extremely healthy in an engineering team. Without a constant low-level feeling of self-doubt you can start to get cocky. Think about that period maybe 2 years into your current profession. You've learned a lot but haven't yet learned enough to understand how dangerous you are. If you have a good amount of self-confidence you'll likely cause a production outage that will naturally instill you with that fear and self-doubt. If not, it might take a bit longer to get there. Obviously everyone is different, and I've certainly met talented engineers that started out and already had this self-doubt. The thing to remember is that it's ok, and can even be helpful to have these feelings.

to new engineers

This may be wrong, but I have a hard time trusting folks that don't experience that fear and doubt. Most of the time it is absolutely not the fault of the engineer, it's just a matter of circumstance. If you are new and are experiencing situations where you feel folks don't trust what you are doing, just remember that a lot of those people have probably been burned in the past by an over-abundance of confidence. To a lot of us this self-doubt is the only thing protecting production.

You can have all the controls in the world in place, but things will always break in production. Knowing that, you have to plan as much as possible and approach solutions with caution and care. If you encounter situations where you think folks don't trust your judgement, take a second and try to have a conversation about what could go wrong with the idea you are proposing; ask others for input. Like with most things, communication is key here. If you can show folks that you have some doubts, and can show how you think you've mitigated the risks, you'll build trust a lot faster.

living with it

Self-doubt is just a part of life as an engineer. It will always be there. Just know that it's normal, and can play an important role in keeping your production systems running. As engineers we should be talking about this, and encouraging conversations about it. Phrases like "I don't know" and "I need to double check that" should be welcome, and if you have doubts they should be expressed and addressed. Resolving your doubts as a team can be a great way to increase trust on your teams and make people more comfortable sharing their ideas. It can also be a great training tool, as the whole team can learn from eachother. So next time you have some doubts, say it out loud (or via chat) and get a conversation started, you and your team will only be stronger for it.