Failure & Change (And more failure)
During this project, I failed a lot. I struggled adding linux accounts, properly managing environments, and getting cron to work properly. While I had done all of these things to various degrees before, I knew I needed to get better at developing full stack applications.
I also at one point accidentally wiped all my data and can honestly say I was heartbroken. When practicing with MySQL, I installed a test database that pushed me over my data limit and with how I had structured my code, it deleted my COVID data to compensate. Luckily I had a backup, but it was missing a few weeks worth.
Ultimately I had to realize that I make mistakes. I fail, a lot. But in the end I have to remember that this is why I’m doing this on my own time, my own projects, and on a $5 a month commitment. Also as weird as it sounds, I just like making things work. As weird as it sounds, I actually slept better the first few days when I knew that 24/7 my code was scraping and storing data on a server with no intervention. Definitely a cool feeling!
For fun, I also include an initial whiteboard sketch of how I initially saw the pipeline working. Obviously it was changed over time, but it was a good first place to start. Also, its the only real photo I had on my phone of this project?