I’m learning that modern (web?) programming is a balance between configuring tools/services and writing your application’s code. The further I get into studying deployments/DevOps methods the more surprised I am that the balance has shifted rather strongly towards configuring tools from third party infrastructure vendors and away from what most would call software engineering1. Why is there such a strong preference for “one step app deployment”, no infrastructure knowledge required?
The obvious answer is because it is easier. We like using high level frameworks to get something pretty in front of users quickly, and most of us aren’t interested (or claim we don’t have time to be interested) in the underlying infrastructure or how it all works.
Infrastructure2 also has a bad reputation. I’ve also read about untold drudgery and headaches from the days of configuring or hosting your own web servers and databases. You have to think of IP addresses and DNS, SSL certificates, Linux distribution updates, security, and on and on. Handing off the complexity is obviously something many (most?) are happy to pay for.
But I’m not from around here– I came to programming and infrastructure from biomedical science, so I’m learning about how lucky I am second hand and dutifully searching stackoverflow and blogs for the magic incantation for enabling SSL for postgres in platform X’s new V2 {app/spec/beanstalk/fly}.yaml file3.
After troubleshooting a database connection for yet another vendor specific yaml file format and realizing the problem was a disagreement in how environmental variables were specified, I stepped back and thought about what I was actually learning in this process, and whether I was getting significant value from using PaaS providers without understanding what they’re actually doing for me.
Acquiring skills vs learning products ¶
If I am working on a personal project and I spend a day figuring out how to configure a not-on-the-happy-path4 option in a vendor specific yaml format, was that time well spent?
For me the answer is no. I’ve just “learned” something that isn’t portable and doesn’t necessarily have any relation to what is really going on with that essential infrastructure3. If I had to start over on a new platform, I’d have to spend that time again.
I remember studying for my AWS Developer Associate exam and feeling like I was actually studying to be an unpaid AWS salesman/developer advocate– always ready with a new AWS product or product combo for every problem. It didn’t feel productive, or like I was actually learning anything that I could build on outside the AWS walled garden. While learning to use something someone else owns is sometimes necessary, it certainly isn’t intellectually rewarding. As a rule, I’d like to minimize the time I spend on non-portable or product focused skills.
More importantly, I realized that if I don’t at least have an understanding of “the hard way” or the default/base case, how can I judge whether these products are a good deal or actually make my life easier? There is a difference between something being difficult because it is unfamiliar and something being difficult due to inherent complexity5.
So I decided to start “doing it myself” for my personal projects6. Reading about the best practices for securing servers and databases, OS updates, logging, firewalls, deployment– all of it. Whenever I start to struggle with a superficially easy PaaS offering, I think: How would I do it if all I had was access to a virtual private server, or an actually private server in my bedroom? And sometimes I’ll try it67. What I’m finding is that while this is a bit more work, it is also a lot more rewarding and fun! I know underneath all the yaml magic, there are real systems, and I’m starting to learn how they interact with one another and work. By peeling back the magic layers, I’m starting to acquire knowledge about the important parts, and I know the time I’m investing in these “fundamentals” will continue compounding and paying off throughout my career89.
You should still use them ¶
This isn’t to say managed options are bad and should be avoided. But I firmly believe that if your career revolves around building these apps, you’d be a much better developer if you at least try to understand the “hard way” too. What are you being paid for anyways? Writing API’s is becoming so routine that AI bots can do it for you.
Your value add is your experience and judgment, and you can’t develop either of those if you’re always catching up to the new vendor’s yaml config format. Plus, I think it demystifies things and leads to better judgment when you do choose to use a PaaS. Most importantly, knowing and learning your tools makes it all a lot more fun.
-
Vicki Boykis knocked this topic out of the park, you should read her post: https://vickiboykis.com/2022/12/05/the-cloudy-layers-of-modern-day-programming/ ↩︎
-
By infrastructure I mean a database, the server(s) to run your database and app, and the associated requirements such as SSL certificates, DNS, etc. ↩︎
-
This post from Terrateam on troubleshooting and trying to figure out how to get SSL on postgres on Fly.io is a great example. Is this better than learning how it is actually done? Fly.io also has a page about custom domains and SSL. Nice docs, but just note that there is so much vendor specific complexity and details here. If you learn how to do it on Fly.io, you’ve consumed huge amounts of details for something that doesn’t apply to anything except this for profit company’s tool. Is that really the best use of your time? ↩︎ ↩︎
-
I consider the happy path as the path that is demonstrated in the README, for example digital ocean’s (DO) app platform spec.yaml. The docs are good, but if your option isn’t directly shown here or in an example it might still be possible, but it isn’t on the happy path. I actually struggled with this when I tried to have DO pull from a private Gitlab repo. I couldn’t figure it out, even though there were instructions for doing it from GitHub. GitLab isn’t on the Happy Path. ↩︎
-
Rich Hickey has some excellent talks on this, see Design, Composition, and Performance and Simple Made Easy ↩︎
-
I started using Ansible as my “cloud vendor agnostic” configuration tool, so I still have some yaml magic. I’ve written about it here, and have a repository with my playbooks. ↩︎ ↩︎
-
I’ve enjoyed using Linode for my learning based VPS’s. They have simple offerings and good pricing. Since the Linode/Akamai rebranding I’ve since switched to AWS and Digital Ocean. ↩︎ -
Or as long as we’re using postgres/MySQL and Linux servers. ↩︎
-
I know there’s “always one level deeper” and I’m still “just configuring” my database or nginx… but it still feels different. So how do you choose which of the seemingly infinite layers to stop at? What about compiling the database with custom settings/optimizations, or going full Gentoo? For me (at the moment) my base layer is the server. I can get servers from any of the big or small cloud providers, and if I can use my tools to run my app from any of them, now that’s powerful. For deeper layers… all in good time! ↩︎