Developer productivity is undergoing a tectonic shift. New software development paradigms and tooling have accelerated the pace and productivity of modern software teams, quickening the "shipping speed" of new software.

To better understand the strategic landscape, my good friend and colleague, Clio Smurro, and I interviewed founders and executives at next-generation software and infrastructure startups pushing the developer productivity frontier to get their thoughts and insights. They shared their views on:

In this third and final chapter, we share our findings on the top challenges and pain points facing developer productivity startups, including:

Subscribe below to receive a nicely formatted PDF of our research!

Defining and planning for success

It's common for developer productivity startups to take an engineering-first approach to development, where engineers take the lead on deciding what to build and when to build it, often leveraging personal intuitions as consumers of the product.

We don't have PMs. We decide what to build democratically. For each sprint, everyone comes up with an idea and we discuss — Manager, Developer Tools Startup

While this works initially, without some sort of formalized product management function, teams soon hit a wall of confusion:

We emulated the Stripe model initially, focusing on engineering and growth. We don't have any PMs and so we don't have a clear framework for what to build — Manager, Developer Tools Startup

It's difficult to chart out long-term priorities and product roadmaps that support the business and revenue generation without product-minded individuals that can liaison between the technical and commercial needs of the company.

If not tackled early, the more haphazard approach to product development can lead to a nonsensical multi-year roadmap of features no one can rigorously justify. With no North Star, new, potentially interesting ideas get shut out and ignored, for lack of space:

We know all the things we would like to build but our roadmap is full — Director, Data Science Startup

To where should developer productivity startups look for signals on what to build next? Customers, ideally. But it's not always that simple. Depending on the customer, signals can be more or less easy to detect. Self-serve users, for example, often silently churn, never indicating what drove the decision:

Self-serve users don’t really tell you what they want, they just churn if they are unsatisfied — Manager, Developer Tools Startup

Telemetry can drive insight into these churn patterns. The problem, however, is that many developer productivity tools are open source, so users typically have the ability to disable usage tracking, cutting off a vital information source. Many tools launch without any tracking in the first place, later causing a riot among their community when implementing telemetry later.

When we can collect user feedback, the next question arises — what to do with it? Should we first solve the problems of our loudest, most-vocal users? Should we focus on features that will help us close revenue in the near-term, perhaps even features that were specifically asked for by prospects? These are all valid questions with no easy answer.

This is a general problem that all dev tools companies run into... building features that specific customers need so that you can close the deal, without any holistic product management framework — Manager, Developer Tools Startup

Though it's somewhat unknowable, developer productivity companies must grapple with and come to a view on the state of the product and whether it's ready for prime-time. One could always collect more data, but eventually you reach critical mass, and decisions can be made. Knowing when you've crossed this key milestone is critical. In other words, what does success look like?

Before reaching this point, it can be hard to tell whether you are on your way or marching in an entirely wrong direction product-wise:

How do you know when you know enough? How do you know when you've seen enough? Are you consensus or non-consensus? If you talk to the market and get 100 "NO"s, but 5 "YES"es, are you right about your idea or are you wrong? — Founder, Data Science Startup

How do you know if you need to change the product geometry? Also, how do you set the right KPIs? — Founder, Data Science Startup

Once product-market fit has been confidently achieved, it's time to pour fuel on the flame and scale up the team. As the team grows, engineering processes inevitably need to change to keep up. This is a key advantages of big tech organizations like Google or Amazon — they've built development processes that work at scale and have been doing so for some time. Developer productivity startups must also set themselves up for success:

Having an effective engineering organization will be a key advantage for us. Big companies like Google and Amazon don’t just have innovative products. They also have innovative engineering processes — Founder, Application Infrastructure Startup

Integrating with other technologies

Few developer productivity startups begin by building an end-to-end solution. Invariably, developer productivity products integrate with other parts of the software development toolchain.

The ML landscape is very messy; very noisy. For the different steps of the ML workflow, people will use the tools which integrate best with each other — Founder, Data Science Startup

Tools should follow UNIX philosophy of working well individually AND integrating with other tools well — Founder, Data Science Startup

It turns out, this is a different skill set from core product building, as many quickly find out:

Seamless integration to our partners technically is challenging. We’re trying to mitigate that by hiring very well-seasoned experts in each of the areas that we’re growing — Founder, Data Science Startup

Integrating with other technologies often involves hiring that particular skill set into the organization. Though it's possible to build integration into tooling that the team doesn't have personal experience with, teams with first-hand knowledge of the complementary technology build the best integrations.

Integration is as much a technological problem as a people problem:

And the human aspect...if you make this huge effort, you want to make sure it’s not just you entering their community, but them really integrating with yours — Founder, Data Science Startup

We need to go one by one in building integrations with each of these open-source communities — Founder, Data Science Startup

Communities already exist around the technologies with which developer productivity startups want to integrate. Ingratiating oneself with these communities is as important as integrating with their tooling.

We’re open source...we have to go into their databases and make sure we get buy-in from their existing developers — Founder, Data Science Startup

Greasing the wheels in this way helps build goodwill among users of the target technologies, which can open doors. The core contributors of the target technology may make an architectural change on their end that makes your integration work much easier. Mutual trust and respect enable this kind of deep collaboration.

Further, people who've spent time in these other ecosystems bring a wealth of knowledge and insights, and can help fledgling developer productivity startups avoid the mistakes of their elders:

Our biggest challenge...we want to learn from each of these communities, and make sure that we’re not repeating any mistakes — Founder, Data Science Startup

Developer productivity startups can't afford to integrate with everything. Resources are limited, and integration work is often not a core competency. Startups must prioritize different integration options. Like sizing up the market opportunity for an entire company, startups should evaluate the market potential of complementary technologies:

Postgres and MySQL….you’ll cover ~70% of the market if you serve those two — Manager, Developer Tools Startup

The proliferation of developer tooling makes it harder make such decisions:

Addressable market for applications is large, but that makes it harder to focus. We need to make sure we’re strategic in choosing which applications to prioritize — Founder, Application Infrastructure Startup

Even among the tools that teams make an affirmative decision to integrate with, quality will vary meaningfully:

We serve like 45 different tools with integration. We do a really good job at serving a few key integrations — Manager, Developer Tools Startup

Building mindshare

Mindshare always precedes dollar share. Potential users make important architectural decisions early, requiring that they know about your technology well ahead of project intiation. Further, up-and-coming vendors must navigate any risk aversion or reticence towards new tooling, which is easier said than done:

Unless developers have the foresight to future proof, they’ll still be building with Postgres or some other older database. We often hear "we'll start with MySQL and the move to you guys if we run into issues" — Executive, Application Infrastructure Startup

Methods for building mindshare are many and varied. Some are subtle. One answer that surprised us but makes complete sense in retrospect? Online classes. Online courses are a common learning vehicle for developers, and the content of most courses naturally tends to skew toward more mature technologies with larger userbases.

This is a sort of network effect whereby MongoDB developers, for example, benefit from the presence of other MongoDB developers because those same developers create an ecosystem of learning content around the database, flattening the learning curve:

An example of mindshare: online courses are teaching other databases, not ours — Executive, Application Infrastructure Startup

If the community isn't large enough, vendors themselves can help cover the gap. Tutorials and documentation help developers both learn how to use a tool but also learn of a tool in the first place. This is classic content marketing, with a developer-focused spin:

One challenge we face is improving our content. Writing tutorials, making videos, helping people use the product better — Manager, Developer Tools Startup

Mindshare has been especially difficult to build during the era of COVID-19, which has put a damper on in-person meetups and other ways in which upstart developer productivity companies build community and evangelism. Startups have rethought more formal events like conferences:

Evangelism has been stymied. The developer ecosystem is very meetup-driven, so that’s been cut — Executive, Application Security Startup

Conferences used to be really important, so we’ll see how being virtual affects that — Developer Advocate, Application Infrastructure Startup

These events often serve as meaningful lead generation channels for developer productivity companies, leading to potential pipeline impacts:

We have a strong pipeline, but we need to make sure we keep that up, especially with everything being virtual now — Developer Advocate, Application Infrastructure Startup

Mindshare, once achieved, can act as powerful social proof. Though many developers claim to be driven by first principles in their design decisions and choice of tools, like all other homo sapiens, they care about what other people think. C-suite buyers care even more about the choices of their peers, so high quality customer logos definitely matter and should be displayed prominently:

We need more logos on our website. Getting named case studies is key for us. CIOs want to know that someone else is using this thing — Executive, Application Infrastructure Startup

Scaling developer relations

Speaking of building mindshare, developer relations (DevRel), also known as developer evangelism, developer advocacy, developer experience, etc. has emerged as an incredible way for developer productivity startups to ramp up mindshare and market awareness. Developers may not like being sold to, but they don't mind nerding out on cool tech with someone who speaks the same language:

Developers don't want to hear from sales folks. They don't want to be told what to do by their CIO. They want to be part of broader community, contribute, participate — Developer Advocate, Analytics Infrastructure Startup

Developers don't like being sold to, but they trust other developers — Developer Advocate, Application Infrastructure Startup

Simple enough. But even if you're fully onboard with the idea of developer relations, doing it well is tough. To start, developer advocates are hard to come by. There just aren't many DevRel folks out there. The function is still relatively new, and career pipelines into the role have yet to fully materialize:

There is a serious lack of talent for DevRel. The ones who are good have already been hired. And the ones who could be good often have a hard time getting a foot in the door without a Twitter presence — Developer Advocate, Application Infrastructure Startup

DevRel is a hard spec to hire for. Technical folks are usually introverts. Extroverts are typically not technical — Developer Advocate, Analytics Infrastructure Startup

Knowing who to hire among available candidates is also tricky. Companies often use social media following as an indicator of DevRel potential. But nearly of the DevRel professionals we spoke to cautioned against using social media following as an rubric for hiring developer advocates, with many saying it's entirely unnecessary for the job. Further, a large social media following is only helpful if it's backed up by technical ability and credibility with the developer community:

You don't really need to have a following to be good at DevRel. It's much more important to have street cred [with developers] — Developer Advocate, Analytics Infrastructure Startup

Social media is just one part of the toolbox. Further, social following in one area does not necessarily translate to another — Developer Advocate, Analytics Infrastructure Startup

There are lots of people with big social media following who no one actually likes — Developer Advocate, Application Infrastructure Startup (author note: 🤣)

Once the DevRel team begins to scale, questions emerge around where it fits within the broader org chart and hierarchy. This quickly gets contentious, and rarely do DevRel teams get to decide their own fate. This leads to bad situations where DevRel ends up in a part of the organization that isn't truly aligned with the practice, or worse, DevRel ends up straddling multiple functions:

Developer relations often gets stuck in the middle between different functions — Developer Advocate, Developer Tools Startup

In the past people didn't really understand DevRel. This lack of understanding drove poor outcomes. Future will be DevRel as its own organization — Developer Advocate, Application Infrastructure Startup

Poor understanding of what DevRel is and its potential drives significant consternation. Again, because there are so few of these individuals and the function is so nascent, identifying enlightened executives and manager to lead these efforts is not trivial. With the right leadership, however, DevRel can thrive:

It's much better to have someone run the team who is involved in developer relations themselves. Having adamant buy in really helps — Developer Advocate, Application Infrastructure Startup

But inevitably the question emerges — how should we measure DevRel's successes (or failures for that matter)?

A lot of people don't know how to be effective in DevRel. You need analytics of some sort. You need to understand your funnel — Developer Advocate, Analytics Infrastructure Startup

This is another point of serious contention within the developer relations community. There are no standard metrics or KPIs in DevRel, leading to difficult conversations within the DevRel team and the rest of the company about whether and how DevRel is pulling its weight:

Metrics are tough. The metrics you end up measuring are often defined by the function you fall under — Developer Advocate, Developer Tools Startup

You will just be looked at as a cost center if you can't quantify this stuff. Really you are revenue generating, and you need to think of yourself that way — Developer Advocate, Analytics Infrastructure Startup

Though there may not be a perfect KPI for DevRel that works for all companies and communities, in picking a set of metrics, it helps to keep in mind a "North Star" tied to the fundamental values and purpose of the business:

Define success by how you serve other developers. Are you giving them what they need so they can build? — Developer Advocate, Application Infrastructure Startup

Conclusion

It was amazing to speak to so many developer productivity founders and operators about the major trends shaping their businesses, their strategic priorities, and their biggest challenges and pain points as they've scaled up. The discussions around challenges were especially humbling. With all the glitz and glamour around startups and Silicon Valley these days, it's easy to forget the fundamentals — building a company in any sector is really, really hard, and developer productivity is no exception.

We hope you've enjoyed this series on developer productivity. We'd love to continue the conversation. If you are a developer productivity founder or operator who's resonated with any of these findings, let's chat!

Clio Smurro & Nnamdi Iregbulem