Product Engineers as Catalysts

TL;DR: Especially in software engineering, the lines between product engineering and product development get blurry. Due to the prevalence of iterative delivery in the development of digital products, organisations that manage to successfully move decision-making closer to or into teams implementing and owning products will reap significant benefits. The Product Engineer can be a significant catalyst in capturing this opportunity from within the team and improve perceivable product quality as well as code quality along the way.

Product Development

Software near always strives to solve the problem of a set of constituents or users. When organisations fund the development of software, they usually develop a product and therefore necessarily engage in some form of product development - whether it be deliberately or incidentally.

Product Engineering

Product development and product engineering are often used interchangeably. Product development includes the full undertaking of creating a new product from ideation through to post-launch activities like marketing. Product engineering, on the other hand, is a narrower subset of activities more concerned with the technical aspects of Product evelopment such as design, development, testing and optimisation.It typically involves several key stages, including conceptualisation, design, prototyping, manufacturing planning, and testing and can be applied to various fields such as software engineering, electronics, industrial equipment, or automobiles.

Product Engineering in Software

While initially, Software Engineering looked towards traditional Engineering and Manufacturing for best practices and methodologies to organise production, it soon became apparent that it's unique combination of attributes (ease of distribution, upgradeability, scalability, etc...) allows for different production and engineering approaches to be explored and new optima to be discovered.
‍
Agile software development commonly refers to the practice of implementing the spiral model for evolutionary software acquisition. It's used by product engineering teams to reach their goals, building products iteratively. As an industry, we've been at this for a while. Dr. Böhm published his article on the spiral model of software acquisition in 1988, the Agile Manifesto was published in 2001, SAFe was first published in 2011 and the European Commission has started to embrace agile practices as recently as 2014. I'll leave it to the reader to decide, which of the aforementioned should really be considered "agile", but the main point we want to drive home here, is that while we, by now, know how we should plan and organise software development successfully, actually building software products successfully and consistently seems to be the dominion of a few.

As alluded to, the proliferation of the spiral model in software is by no means accidental. The inherent attributes of software enable this staggered creation of value, generally decreasing time to market and time to feedback - which is great! The accelerated cadence of delivery, in turn, leads to an increased number of journeys through the stages of product engineering which comes with a new set of challenges. One of those being that organisations built on top of highly specialised functions have to go through more handovers until value is delivered. This already is a problem in other industries, but it surely is a problem in Software. Handovers are inherently expensive because they always carry a risk of misalignment and work on moving an item down the value chain to the customer usually doesn't continue without delay. Bottom line: Handovers are expensive.

This is nothing new. We've been feeling that pain for years and have become quite adept at dealing with it. We achieved this by both a shift-left and a shift-right in the industry to reduce the number of handovers and increase our agility with it - terms like Full-Stack Engineers and DevOps come to mind. This has generally had positive impacts on the efficiency and effectiveness of the software development process.

In my opinion, the next frontier of improvement in product engineering lies in establishing better alignment and stronger ties between product development and product engineering to decrease our reaction times and avoid costly miscommunication.

The above is especially true when we're in context's with more uncertainty than a "simple" project delivery of known value and scope. We have our "ear to the market" and strive to get a lot of feedback quickly. In such contexts, we release often and build iteratively to de-risk our product development efforts. A good amount of releases might contain hypotheses such as "Would reduced latency really drive engagement?" or "Will a bulk-editing feature be adopted and therefore deemed useful by our user?".

If more development effort can be seen as a late-stage experiment, we would benefit if the Product Engineering Team had the capabilities to own and evaluate the outcome of their efforts. Mind you, this doesn't only concern what's being built, but also how we build it. As Engineers we know that a highly experimental feature might be built and rolled out differently than one we know will be used by 90% of users and needs to scale to hundreds of thousands of requests per day.

We're starting to see that product development often has to "sense" through Engineering and that a lot of Product context will influence how exactly features are being built and evaluated. So if we know that we'd like to increase the bandwidth and collaboration between these functions, how could we achieve it?

Product Engineers

There have always been a few standout engineers who strove to really understand the domain and the business. They tend to do be hard to find and exceptionally valuable to their employers. Instead of leaving stumbling on them up to chance, we can try to capture that lightning in a bottle by drawing the outlines of a role which we will call the "Product Engineer" for our purposes here.

What are the defining characteristics of this Product Engineer? The term is gaining traction and loosely applied throughout different opinion-pieces and hence is in a state of flux. Definitions range from individuals that conceptualise, design, implement, run and potentially even market a whole product on one end to "Product-Minded Engineers" which clearly remain in Engineering but are inherently curious and understanding of surrounding functions on the other. I believe that it boils down to a tradeoff optimising for added value through higher bandwidth and internalised product and organisational knowledge without sacrificing performance in one's engineering capabilities. Pair that with prioritising outcomes over output and you have a potent combination that can slot into most organisations.‍

I am wary of suggestions that stray too far into the "one man digital product army" definition of the Product Engineer as a concept. I've spent the early days of my career in Strategy and Business Development and have a lot of respect for the work being done there and in other functions of the value chain. To do them well, you can't do them on the side. You likely haven't seen many Engineering Managers pull a couple of tickets each sprint in between one-on-ones either - if you have, how has that worked out? What is more, cognitive load theory and the fact that excessive task switching leads to significant decreases in productivity (as much as 40%) strongly suggests that spreading yourself thin over many areas of responsibility and skills likely doesn't improve outcomes.‍

All that being said, I have personally made good experiences with jumping on tasks bottlenecking value delivery despite them being outside my responsibilities. If nobody does exploratory interviews with customers what you're building likely isn't the most impactful. Likewise, fully implementing new complex features without first testing them with any users probably is a suboptimal use of engineering capacity or leads to unnecessarily expensive iterations. Yet if demands for capacity or quality should exceed what's available from the Product Engineer - these gaps should be filled. An excessive amount of breadth in responsibilities and attention can only humanly possibly come at the expense of technical excellence - which I consider a requirement for being a great Product Engineer. It is a requirement for freeing up enough time and mental capacity to concern oneself with more.

To sum it up, in my view, a Product Engineer...

• ... excells in his/her Software Engineering craft
• ... focuses on outcomes over output
• ... has curiosity and empathy for users and collaborators
• ... is curious about the business and value delivery
• ... is proactive and brave enough to fill in adjacent bottlenecks
• ... prefers data over opinions

Conclusion

Over the course of the last 70 years the goal posts for what make a piece of software a good product shifted tremendously. The pace of this shift, or more precisely the volatility of markets, customers and needs continues to increase. We colloquially accept that we live in a VUCA world, but our industry has only gradually come to fully take this into account and strengthened the ties between functions involved in digital value delivery.

Strongly integrating product-thinking into development teams improves the speed and quality of iterations and minimises alignment-gaps such that all effort is expended with increased precision. The result should be product developed with a much more resilient and higher-bandwidth connection to the market and throughout the whole production process.

Dev-Teams don't need to consist entirely of Product Engineers, but where there are none or too few, they can generate outsized value.

* illustrations in this article are intentionally overly simplified so as not to distract from the heart of the article. A lot of nuance is lost in that simplification - I am aware of that.