Building things right makes everything else easier

It's very tempting to build something really quick, ship it out there, and start growing quickly. Sacrificing some quality for speed may make sense while building a prototype at the idea validation phase. That said, once the idea is validated, the product needs to be properly architected for scale. The reason behind it is that you need to set up your product architecture for future growth. If your product is not built for scale in mind, and you try to scale it, your technology will eventually stop supporting the increased traffic. As a consequence, your code will be convoluted to be able to make future changes rapidly, the team morale will go down which can lead to losing your core team members. Time and time again we’ve seen companies not architecting things properly at the beginning. Then needing to hire more engineers later to support an unscalable system. Long story short, if you build your product right in the beginning, scaling it won’t be a problem. Your cloud infrastructure bill will be low and you won’t need that many engineers to support it and grow it.

A classic example of not building for scale is starting with a monolithic architecture, putting all your code in a single application, then taking it too far to later realize that you need to rebuild the entire product. A better way is to start with the microservices architecture from the beginning (or at least pivot into microservices as the application gets large). This way you develop your software in multiple modules with a clear set of responsibilities. This approach is a lot more manageable and more scalable because you can scale each module independently. Here is a link to the Serverless Microservices solution going deeper on the topic.

In the meantime, let’s go over the 10 ways to build a long-lasting product:

1. Achieve Company Alignment

It’s important to have a buy-in from the whole organization that rushing through and shipping low-quality product is counterproductive. Each shortcut we take will add on to our technical debt which needs to be paid off. On the other hand, there should always be urgency around shipping because it’s easy to fall into a trap of building a perfect product for a long time. The best code is worthless if it’s not shipped and running in production without any issues. It’s all about finding the right balance (no pun intended).

2. Minimize System Downtime

As software engineers, we should strive to build computer systems that are always up and serving our customers without our manual intervention. Why? Because it’s in the nature of our profession and in the core strength of the whole of humankind to build systems that work without requiring our presence. While developing a software component, engineers should always be focused on developing it in a way so it can deliver customer value and recover gracefully from an error situation. Which involves performing validation on the incoming data, making sure there are self-healing automated systems in place. And if the error happens, it’s clear what to do, appropriate people are notified for taking care of the issue.

3. Minimize Maintenance

Doing things right is hard and it takes time. However, if you commit to doing things right, then eventually, doing things right becomes second nature. In addition to it, doing things right becomes easier the more you do it. So, if there is a question when you should start doing things right, the answer is now. It’s very easy to fall into a trap of introducing a couple of manual steps in your deployment or some SQL you need to run sometimes to clean up your data. The goal is to reduce the number of manual steps engineers need to do instead of adding more manual steps. Each maintenance step you add may not seem like a lot. It’s a slippery slope that turns software engineers into maintenance engineers. We’ve seen the entire engineering team not producing any more new code and just doing some manual tasks to keep the product running. Unfortunately, it’s quite common, especially in large organizations. All high maintenance systems start with just one manual step you leave behind.

4. Automate Manual Work

The best use of a software developer’s time is to create systems that deliver value. Coding is hard and there are only so many hours in a day. So it’s best to apply our coding skills towards automating some manual work we otherwise need to do. Another issue with manual work is that it requires a context switch. Because coding requires you to deal with a lot of variables, it takes some time to get into the context (what some people call getting into the zone). Switching to a manual task, like updating some records in the database, requires you to switch your context. Then once the task is done, coming back to coding will take some time again because you need to get into another, more complicated context of coding. If you have several manual tasks spread throughout the day, you’ll find yourself not producing any meaningful code by the end of the day. That said, doing some manual work once in a while is unavoidable. The solution is to strive to minimize it and automate it away instead of adding to it.

5. Keep your hand on the pulse with Alerting and Monitoring

Each microservice or a client application has to have some alerting and monitoring component before going live. Your component may work on your local machine or even in the hands of a QA person. Once it goes live, the only way to know if it works as expected after you deploy your changes is to have some comprehensive alerting and monitoring. If an error happens, it should be sent to email/text/slack/PagerDuty or anywhere it can be seen and acted upon. The goal of it is to not create more distractions or more manual work, but rather to find if we missed something that should be added as a feature or an improvement to the product. As long as you keep on improving and refactoring your system, you’ll one day come to a place where things just work, always.

6. Measure and Analyze

If you can’t measure it, you can’t improve it.

• Peter Drucker

Trusting your gut is important, that said, business decisions need to be made based on some real data. Our recommendation is to target a specific set of metrics for the product, collect the data, and monitor the metrics closely with some regular analysis. Some of the metrics can be collected from the client application like button clicks, page views, etc. And some need to be collected from the back end. The best solution is to have a proper data warehouse or data lake where all sources of data come together in a structured format you can query. This allows you to connect the dots and build conversion funnels based on the rich set of data.

7. Document instead of Keeping Things in Your Head

Having the system knowledge only in someone's head is risky. It decreases your bus factor. If the knowledge is spread out then having to talk to multiple people to understand how the system works is inefficient. It also slows down your team velocity in the long run. The solution to this problem is simple. You choose your documentation tool (Quip, Confluence, or even Google Docs work just fine) and you make it a good practice to document how each part of the system works. Once the modification to the system is made, the documentation needs to be updated too. This way, there is a consistent process for ramping up new developers on any part of your system. New developers just read the existing docs and make the updates as necessary.

8. Minimize Management Overhead

It’s amazing to see how a lot of open source projects streamline the process for reporting issues, implementing the fixes, and rolling out new releases. The same principles should apply within your organization too. The key to minimizing management overhead is to have a solid feature delivery process (preferably Agile) that everyone believes and follows. Things like introducing more changes mid-sprint create additional scope-related conversations, change of context, sometimes misaligned expectations. It takes energy to address them, and this energy could be spent on delivering the functionality by sticking to the plan.

9. Maximize Maker Time

Maker’s Schedule, Manager’s Schedule article from Paul Graham, one of the founders of YCombinator describes the difference between how the maker operates and the manager operates. It’s important to realize that your product is built by makers. Software engineering is hard and it consumes a lot of brain energy. To make a great product, it takes some time to get into the zone. And if a developer gets constantly dragged into some side conversations and meetings, getting into the zone is almost impossible. Maximizing maker time, removing the meetings and distractions from your developers is key to having a high velocity on your team.

10. Have Fun!

Software engineering is a creative process. Any creative process requires having fun, having high energy fueled by enjoying what you’re doing. Just cranking through some boring work isn’t going to get your team very far in the long run. Doing things so it’s exciting, using the latest technologies, solving problems in new and creative ways is what makes the products great. If things get boring, think about how you can spice things up using the new tech, features, or product opportunities. You’ll get a highly motivated team having a lot of fun!

Conclusion

There is no time to waste building your product. Everything you build today can last decades and affect millions of people. When the basics are done right, scaling your product is easy. It lowers your maintenance costs so you can focus on future growth and development. Doing so requires some effort upfront which is totally worth it.

So what are you waiting for? Go ahead and build something that scales and lasts! 🙌