Re-Inventing The Wheel

There’s a saying well known by most programmers: “Don’t reinvent the wheel’. It is said in many scenarios, but means the same thing in almost all of them: Learn from those that came before you, and use their solutions to these problems, instead of wasting your time trying to do the same thing again.

While building my app, Caelum, I had to search high and low for packages (self-contained, pre-written code) that could handle the networking and server communication side of things — talking with servers, parsing and validating data payloads, maintaining security, etc. — but alas, for reasons I will be talking about soon enough, I could not find one that had all the features I wanted : Security, Reliability, Modularity and Scalability.

After hours of fruitless searching, I finally came to the conclusion that, weak as I may have been with network code, I was going to have to do this myself.

The list of requirements was long and arduous: I had to design a robust API request system capable of handling a wide variety of request types, converting each instance of numerical data to and from a plethora of units, also accounting for every error the server could throw at me, not to mention implementing fallbacks upon fallbacks, as this system would lie at Caelum’s heart and would be what drove everything else. So it had to be perfect, or at least damn near it.

And it had to be so on the first try, as I didn’t have the time required to be able to extensively test every edge case and properly trigger every error to check if the system could handle it. This was going to be long and difficult, to say the least.

These intimidating specs, however, far from actually making me frustrated, actually got me even more excited to jump into the code. You see, I wouldn’t have fallen in love with code as much as I have if I didn’t like a good challenge and solving complex problems, after all.

Working with code, you come to appreciate not just a challenge at face value, but also learn to delve deep into its nuances, dance across the intertwined logic that makes up the problem and might just make up the solution, and in doing so you gain perspective that is invaluable.

This system was seeming more and more like a challenge of that same caliber (and it indeed would turn out to be so) and so, fortunately, the excitement that came with the problem at hand went far in alleviating the stress of the problem at hand.

This excitement also didn’t just come from the hint of a unique challenge, but also in the form of the coolness of certain features I wanted the system to have, such as the MJP.

I had wanted to implement a custom (and awesome) security measure, the Module Jettison Protocol, where if a fatal error occurs at any step of the process, the system will just shut that part off and return diagnostic information instead of failing.

The name I gave it alone was cool enough to warrant at least an evening trying to make this, and although it did take a bit longer to really get it just right, I’m proud of the end result and think the name sounds even cooler now that its backed by actual functioning code.

It did not, of course, feel like it while coding, but looking at it now, the entire system has grown to be quite formidable and quite powerful, with nearly the entire API endpoint having been mapped to options in its querying capabilities, all of it also fully secure and guarded against any error.

On The Shoulders of Giants

This is not to say however, that the credit is all mine. No, a part of this belonged to every single one of the huge number of open source software contributors out there, whose generosity and selflessness, combined with huge skill and a desire to give back was the reason I could build this at all, not to mention within the short timeframe that I did.

So, when I saw that this system had grown so much, and was better than a lot of the implementations currently out there, I decided then, that I would be open sourcing it as a package on pub.dev.

Of course, this would take some work, properly generalizing its functions, exhaustively documenting the different classes, reducing its dependencies, isolating and severing its connections to Caelum’s codebase, etc.

Not too tough, but all of it things that did have to be done before I could publish it. I also had to write quite a bit of nice literature (among other things, a README.md file, a CHANGELOG.md file and a LICENSE file) for it, and like any nice thing, those would take time to make.

The Start

I started work on this mini-project the day after Caelum launched, and about a day later, I was ready to publish a V1.0.0. My previous experience publishing MobX-Architecture made it so that I only had to focus on the code this time around, as I already had adequate knowledge on how the publishing process worked, which greatly expedited progress on this package.

The package of course still has some quirks I’d like to iron out, and still has a few features left to implement to really be able to query all the API endpoints served by OpenWeather, as, since Caelum didn’t require querying those particular endpoints, the code for it never got written.

Still, I’m proud of where the package is currently, and the literature associated with it, and plan to add the endpoints it is currently missing as soon as I have time.

The Why?

Flutter, the framework I am building my app, Caelum, in, is, relatively speaking, quite new when you compare it to its native counterparts, and downright in its infancy when you compare it to something like web development using HTML, CSS and JS.

That is after all why I had to build this system, as the relative newness of Flutter means a package providing this functionality just hasn’t been created yet. That of course also raises the question, then why am I using Flutter in the first place ?

The answer to that, has to do with several usually unrelated things: A child’s wonder, the Wild Wild West, and monuments.

Let me explain.

The Old, The New, and That Which Is Wondrous

Working with well established technologies has a plethora of well known benefits, ranging from an absolutely gargantuan number of tutorials, guides and open sourced code to aid you (something I experienced first hand when I built my website, the article to which you can check out here) to the copious amounts of ready help you can receive on sites like StackOverflow, since the tech’s long lifespan has resulted in an overwhelming chance that someone else has not only experienced the error you are facing right now, but has also fixed it, posted solutions and guides on it, and perhaps even started an open source project surrounding it to make sure no one ever has to face it ever again.

And, from a practical point of view, there aren’t even many disadvantages. A perfect thing, if there ever was one.

Now that is not to say however, that newer technologies don’t have advantages. Far from it actually, newer technologies bring with them fresh perspective and a different way to do things, not to mention a plethora of features from faster compile times to solutions to the pitfalls of their predecessors.

They are, of course, also more in touch with the requirements of today, and when fleshed out can serve it better than older technology ever could, regardless of how much they have been updated, as, after all, equivalent older technologies are fundamentally different at their cores, having been built for a different time and therefore different requirements and specifications.

But newer approaches are also plagued by the issues surrounding any relatively new technology: bugs, a small community, undocumented errors etc.

Not so perfect. At least, not yet.

Now why mention all of this? You see, I’m all about using the most efficient method of doing something, and that, more often than not, would mean using older, established technology, but, childish though it may seem, there’s something about the charm of new technology, not only its touted benefits or faster compile times or even developer friendliness, but rather the allure of uncharted territory, the allure of stepping into the unknown, and of course, the allure of building something in this new frontier that will last the test of time, that drives me to it more often than not.

You see, for older technology, the ‘mystery’ so-to-speak, has faded, and with it, some of the magic too. Packages and projects have been built to aid in every endeavor, bridges have been built in code to traverse the deepest ravines, and the highest mountains have had roads paved all the way to their tops, and while it of course makes building anything on top of it trivially easy, The Age of Explorers as it were, has already come and gone for such technologies, and there’s not much to build for it. But not so for their newer counterparts.

For their newer counterparts, the landscape is akin to the Wild Wild West. Now that may be the cause of most if not all of the problems mentioned above, but, for me at least, it also brings a sense of that long forgotten mystery. It brings with it an air of magic, in a new land of opportunity.

It brings the possibility, that any thing you build becomes a bridge, becomes that road painstakingly carved into the mountainside, becomes that awe inspiring tunnel bored straight through its heart and out the other side: infrastructure and tools used by and helpful to hundreds if not thousands down the line, as the technology grows.

But why this enthrallment with the unknown, with building, and this obsession with the air of magic surrounding technology, you may ask? Well, the answer is two fold.

You see, when I was much younger, I’d look around at all of the electronics surrounding me, things we more often than not take for granted, from the television in the living room to the oven in the kitchen, and I vividly remember the infinite sense of bewilderment and wonder that would consume my mind when I tried to understand or predict how they worked (and later on when I would take them apart to see for myself).

It was Magical. All of it. Right down to each and every transistor and each and every line of code.

Understanding the inner workings of things (particularly electronics), or at least trying to, has always been one of my passions, sometimes even to a flaw. I am, however, glad to say that, in coding however, it has been a boon unlike any other.

Learning about computers, both hardware and software, wasn’t always an easy or even rewarding process, particularly near the beginning, when nearly every aspect of it was overwhelming.

But in those times, with my patience waning and frustration taking hold, my burning desire to truly get how the computer I was searching those very things on worked, why they worked, and how I may make them do something else, that was what kept me going.

For, only if I truly understood this system of systems could I start moulding it, start wielding some of the magic that made it up, making it follow my directions, turning my ideas into reality.

And now, all these years later, exploring the limits of these new technologies, building projects with them, once more turning my ideas into tangible results, does the same.

It brings the mystery back, the sense of wonder, the magic and most importantly, the feeling that I could build something meaningful here, something new, something useful, something to become one of the vast and useful softwares of the future.

The End ?

This desire to explore the unknown, to wield its magic and help build a better world with it, of course, is not limited to me, after all it took us from living in caves and scavenging for food to building cities whose glow rival that of the stars, and farms whose scale and complexity is staggering at its simplest.

It took us from being mere admirers of the cosmos to trained users of its deepest intricacies, wielders of the primordial forces of electricity, magnetism and the nuclear forces that make it up.

It took us from looking up at the sky in wonder to building complex automata that ride flames created from our prowess in chemical engineering to send probes into the vastness of space engraved with and named after the traits ingrained in our very DNA, Spirit, Curiosity, Perseverance, etc.

It is, to say the least, part of who we are.

And so, dear reader, I leave you to this:

Explore. The deepest caverns and the tallest mountains. Be it cyberspace, physical space, or space itself.

Build. Create. And most importantly, Give Back. For, if we have seen further, it is by standing on the shoulders of giants.

Thank you for reading this post, and have a wonderful day!