No more magic, please!

I’m learning that modern (web?) programming is a balance between configuring tools/services and writing your application’s code. The further I get into studying deployments/DevOps methods the more surprised I am that the balance has shifted rather strongly towards configuring tools from third party infrastructure vendors and away from what most would call software engineering¹. Why is there such a strong preference for “one step app deployment”, no infrastructure knowledge required?

The obvious answer is because it is easier. We like using high level frameworks to get something pretty in front of users quickly, and most of us aren’t interested (or claim we don’t have time to be interested) in the underlying infrastructure or how it all works.

Infrastructure² also has a bad reputation. I’ve also read about untold drudgery and headaches from the days of configuring or hosting your own web servers and databases. You have to think of IP addresses and DNS, SSL certificates, Linux distribution updates, security, and on and on. Handing off the complexity is obviously something many (most?) are happy to pay for.

But I’m not from around here– I came to programming and infrastructure from biomedical science, so I’m learning about how lucky I am second hand and dutifully searching stackoverflow and blogs for the magic incantation for enabling SSL for postgres in platform X’s new V2 {app/spec/beanstalk/fly}.yaml file³.

After troubleshooting a database connection for yet another vendor specific yaml file format and realizing the problem was a disagreement in how environmental variables were specified, I stepped back and thought about what I was actually learning in this process, and whether I was getting significant value from using PaaS providers without understanding what they’re actually doing for me.

Acquiring skills vs learning products ¶

If I am working on a personal project and I spend a day figuring out how to configure a not-on-the-happy-path⁴ option in a vendor specific yaml format, was that time well spent?

For me the answer is no. I’ve just “learned” something that isn’t portable and doesn’t necessarily have any relation to what is really going on with that essential infrastructure³. If I had to start over on a new platform, I’d have to spend that time again.

I remember studying for my AWS Developer Associate exam and feeling like I was actually studying to be an unpaid AWS salesman/developer advocate– always ready with a new AWS product or product combo for every problem. It didn’t feel productive, or like I was actually learning anything that I could build on outside the AWS walled garden. While learning to use something someone else owns is sometimes necessary, it certainly isn’t intellectually rewarding. As a rule, I’d like to minimize the time I spend on non-portable or product focused skills.

More importantly, I realized that if I don’t at least have an understanding of “the hard way” or the default/base case, how can I judge whether these products are a good deal or actually make my life easier? There is a difference between something being difficult because it is unfamiliar and something being difficult due to inherent complexity⁵.

So I decided to start “doing it myself” for my personal projects⁶. Reading about the best practices for securing servers and databases, OS updates, logging, firewalls, deployment– all of it. Whenever I start to struggle with a superficially easy PaaS offering, I think: How would I do it if all I had was access to a virtual private server, or an actually private server in my bedroom? And sometimes I’ll try it⁶⁷. What I’m finding is that while this is a bit more work, it is also a lot more rewarding and fun! I know underneath all the yaml magic, there are real systems, and I’m starting to learn how they interact with one another and work. By peeling back the magic layers, I’m starting to acquire knowledge about the important parts, and I know the time I’m investing in these “fundamentals” will continue compounding and paying off throughout my career⁸⁹.

You should still use them ¶

This isn’t to say managed options are bad and should be avoided. But I firmly believe that if your career revolves around building these apps, you’d be a much better developer if you at least try to understand the “hard way” too. What are you being paid for anyways? Writing API’s is becoming so routine that AI bots can do it for you.

Your value add is your experience and judgment, and you can’t develop either of those if you’re always catching up to the new vendor’s yaml config format. Plus, I think it demystifies things and leads to better judgment when you do choose to use a PaaS. Most importantly, knowing and learning your tools makes it all a lot more fun.