Scaling Engineering

I. Maintaining engineering velocity as you scale

By Marcelo Cortes, Cofounder/CTO, Faire

Engineering is typically the function that grows fastest at a scaling startup. It requires a lot of attention to make sure the pace of execution does not slow and cultural issues do not emerge as you scale.

We’ve learned a lot about pace of execution in the past five years at Faire. When we launched in 2017, we were a small and nimble team of five engineers. From the beginning, we built a simple but solid foundation that allowed us to maintain both engineering velocity and quality. When we found product-market fit later that year and started bringing on lots of new customers, we were able to grow even faster, rather than having to re-architecture the platform. In this post, we discuss the guiding principles that allowed us to maintain our engineering velocity as we scaled.

Four guiding principles to maintaining velocity

Faire’s engineering team grew from five to over 100 engineers in three years. Throughout this growth, we were able to sustain our pace of engineering execution by adhering to four important elements:

1. Hiring the best engineers
2. Building solid long-term foundations from day one
3. Tracking metrics to guide decision-making
4. Keeping teams small and independent

1. Hire the best engineers

You want to hire the best early team that you can, as they’re going to be the people helping you scale and maintain velocity. And good people follow good people, helping you grow your team down the road.

This sounds obvious, but it’s tempting to get people in seats fast because you have a truckload of priorities and you’re often the only one doing engineering recruiting in those early years. What makes this even harder is you often have to play the long game to get the best engineers signed on. Your job is to build a case for why your company is the opportunity for them. 

We had a few amazing engineers in mind we wanted to hire early on. I spent over a year doing coffee meetings with some of them. I used these meetings to get advice, but more importantly I was always giving them updates on our progress, vision, fundraising, and product releases. That created FOMO which eventually got them so excited about what was happening at Faire that they signed up for the ride.

While recruiting, I looked for key competencies that I thought were vital for our engineering team to be successful as we scaled. These were:

a. Experts at our core technology

In early stages, you need to move extremely fast and you cannot afford to make mistakes. We wanted the best engineers who had previously built the components we needed so they knew where mistakes could happen, what to avoid, what to focus on, and more. For example, we built a complex payments infrastructure in a couple of weeks. That included integrating with multiple payment processors in order to charge debit/credit cards, process partial refunds, async retries, voiding canceled transactions, and linking bank accounts for ACH payouts, etc. We had built similar infrastructure for the Cash App at Square and that experience allowed us to move extremely quickly while avoiding pitfalls.

b. Focused on delivering value to customers

Faire’s mission is to empower entrepreneurs to chase their dreams. When hiring engineers, we looked for people who were amazing technically but also understood our business, were customer focused, were passionate about entrepreneurship—and understood how they needed to work. That is, they understood how to use technology to add value to customers and product, quickly and with quality. To test for this, I would ask questions like: “Give me examples of how you or your team impacted the business.” Their answers would show how well they understood their current company’s business and how engineering can impact customers and change a company’s top-line numbers.

I also learned a lot when I let them ask questions about Faire. I love when engineering candidates ask questions about how our business works, how we make money, what our market size is, etc. If they don't ask these kinds of questions, I ask them things like: “Do you understand how Faire works?” “Why is Faire good for retailers?” “How would you sell Faire to a brand?” After asking questions like these a few times, you’ll see patterns and be able to quickly identify engineers who are business-minded and customer-focused.

Another benefit of hiring customer-focused engineers is that it’s much easier to shut down projects, start new ones, and move people around, because everyone is focused on delivering value for the customer and not wedded to the products they helped build. During COVID, our customers saw enormous change, with in-person trade shows getting canceled and lockdowns impacting in-person foot traffic. We had to adapt quickly, which required us to stop certain initiatives and move our product and engineering teams to launch new ones, such as our own version of online trade shows. 

c. Grit

When we first started, we couldn’t afford to build the most beautiful piece of engineering work. Instead, we had to be fast and agile. This is critical when you are pre-product-market fit. In the early days, our CEO Max and a few early employees would go to trade shows to present our product to customers, understand their needs, and learn what resonated with them. Max would call us with new ideas a few times a day. It was paramount that our engineers were gritty and able to quickly make changes to the product.. Over the three or four days of a trade show, our team deployed changes nonstop to the platform. We experimented with offerings like:

• Free shipping on first orders
• Buy now, pay later
• Buy from a brand and get $100 off when you re-order from the same brand
• Free returns

By trying different value propositions in a short time, our engineering team helped us figure out what was most valuable to our customers. That was how we found strong product-market fit within six months of starting the company.

2. Build a solid long-term foundation from day one

The number one impediment to engineering velocity at scale is a lack of solid, consistent foundation. A simple but solid foundation will allow your team to keep building on top of it instead of having to throw away or re-architecture your base when hyper growth starts.

To create a solid long-term foundation, you first need to get clear on what foundations you believe are important for your engineering team to scale. For example, I remember speaking with senior engineers at other startups who were surprised we were writing tests and doing code reviews and that we had a code style guide from the very early days. But we couldn’t have operated well without these processes. When we started to grow fast and add lots of engineers, we were able to keep over 95% of our engineering team focused on building features and adding value to our customers, increasing our growth. 

Once you know what long-term foundations you want to build, you need to write it down. We were intentional about this from day one and documented it in our engineering handbook. Today, every engineer is onboarded using this handbook.

The four foundational elements we decided on were:

a. Being data-driven

The most important thing is to build your data muscle early. We started doing this at 10 customers. At the time, the data itself wasn’t particularly useful; the more important thing was to start to collect it. At some point, you’ll need data to drive product decision-making. The longer you wait, the harder it is to embed into your team.

Here’s what I recommend you start doing as early as possible:

• Set up data pipelines that feed into a data warehouse.
• While you’ll largely be relying on intuition to build the first version of your product, you should start collecting data on how people are using it. As you add features and iterate, you should also record how those changes are impacting user interactions. All of this should go into a data warehouse that is updated within minutes and made available to your team. As your product gets increasingly complex, it will become more and more important to use data to validate your intuition.
• We use Redshift to store data. As user events are happening, our relational database (MySQL) replicates them in Redshift. Within minutes, the data is available for queries and reports.
• Train your team to use experimentation frameworks in product development. 
• Make it part of the product development process. The goal is to transform your intuition into a statistically testable statement. A good place to start is to establish principles and high-level steps for your team to follow when they run experiments. For example, we’ve set principles around when to run experiments vs. when not to, that running rigorous experiments should be the default (and when it isn’t), and when to stop an experiment earlier than expected. We also have teams log experiments in a central dashboard in Notion.
• The initial focus should be on what impact you think a feature will have and how to measure that change. As you’re scoping a feature, ask questions like: How are we going to validate that this feature is achieving the goals we intended? What events/data do we need to collect to support that? What reports are we going to build? Over time, these core principles will expand.
• The entire team should be thinking about this, not just the engineers or data team. We reinforced the importance of data fluency by pushing employees to learn SQL, so that they could run their own queries and experience the data firsthand.
• It’ll take you multiple reps to get this right. We still miss steps and fail to collect the right data. The sooner you get your team doing this, the easier it will be to teach it to new people and become better at it as an organization.

b. Our choice of programming language and database

When choosing a language and database, pick something you know best that is also scalable long-term. If you choose a language you don’t know well because it seems easier or faster to get started, you won’t foresee pitfalls and you’ll have to learn as you go. This is expensive and time-consuming. We started with Java as our backend programming language and MySQL as our relational database. In the early days, we were building two to three features per week and it took us a couple of weeks to build the framework we needed around MySQL. That was a big tradeoff that paid dividends later on. 

c. Writing tests from day one

Many startups think they can move faster by not writing tests; it’s the opposite. Tests allow you to move faster, avoid bugs, and prevent legacy code at scale. They aren’t just validating the code you are writing now. They should be used to enforce, validate, and document requirements. Good tests protect your code from future changes as your codebase grows and features are added or changed. They also catch problems early and help avoid production bugs, saving you time and money. Code without tests becomes legacy very fast. Within months after untested code is written, no one will remember the exact requirements, edge cases, constraints, etc. If you don’t have tests to enforce these things, new engineers will be afraid of changing the code because they won’t know if they broke something or changed an expected behavior.

There are two reasons why tests break when a developer is making code changes:

• Requirements change. In this case, we expect tests to break and they should be updated to validate and enforce the new requirements.
• Behavior changes unexpectedly. For example, a bug was introduced and the test alerted us early in the development process.

Every language has tools to measure and keep track of test coverage. I highly recommend introducing them early to track how much of your code is protected by tests. You don’t need to have 100% code coverage, but you should make sure that critical paths, important logic, edge cases, etc. are well tested. Here are tips for writing good tests.

d. Doing code reviews

We started doing code reviews when we hired our first engineer. Having another engineer review your code changes helps ensure quality, prevent mistakes, and share good patterns. In other words, it’s a great learning tool for new and experienced engineers. Through code reviews, you are teaching your engineers patterns: what to avoid, why to do something, the features of languages you should and shouldn’t use. 

Along with this, you should have a coding style guide. Coding guides help enforce consistency and quality on your engineering team. It doesn’t have to be complex. We use a tool that formats our code so our style guide is automatically enforced before a change can be merged. This leads to higher code quality, especially when teams are collaborating and other people are reviewing code.

We switched from Java to Kotlin in 2019 and we have a comprehensive style guide that includes recommendations and rules for programming in Kotlin. For anything not explicitly specified in our guide, we ask that engineers follow JetBrains’ coding conventions.

These are the code review best practices we share internally:

• #bekind when doing a code review. Use positive phrasing where possible ("there might be a better way" instead of "this is terrible"; "how about we name this X?" instead of "naming this Y is bad"). Especially working remotely, it's easy to unintentionally come across as harsh.
• Don't block changes from being merged if the issues are minor (e.g., a request for variable name change, indentation fixes) as this might come across as harsh. Instead, make the ask verbally. Only block merging if the request contains potentially dangerous changes that could cause issues or if there is an easier/safer way to accomplish the same.
• When doing a code review, ensure that the code adheres to your style guide. When giving feedback, refer to the relevant sections in the style guide.
• If the code review is large, consider checking out the branch locally and inspecting the changes in IntelliJ (Git tab on the bottom). It’s easier to have all of the navigation tools at hand. 

3. Track metrics to drive decision-making

Tracking metrics is imperative to maintaining engineering velocity. Without clear metrics, Faire would be in the dark about how our team is performing and where we should focus our efforts. We would have to rely on intuition and assumptions to guide what we should be prioritizing. 

Examples of metrics we started tracking early at Faire (at around 20 engineers) included:

• Uptime. One of the first metrics we tracked was uptime. We started measuring this because we were receiving anecdotal reports of site stability issues. Once we started tracking it, we confirmed the anecdotal evidence and dedicated a few engineers to resolve the issue.
• CI wait time. Another metric that was really important to us was CI wait time (i.e., time for the build system to build/test pull requests). We were receiving anecdotal reports of long CI wait times for developers, confirmed it with data, and fixed the issue.

This is a dashboard we created in the early days of Faire to track important engineering metrics. It was updated manually by collecting data from different sources. Today, we have more comprehensive dashboards that are fully automated.

Once our engineering team grew to 100+, our top-level metrics became more difficult to take action against. When metrics trended beyond concerning thresholds, we didn’t have a clear way to address them. Each team was busy with their own product roadmap, and it didn’t seem worthwhile to spin up new teams to address temporary needs. Additionally, many of the problems were large in scale and would have required a dedicated group of engineers. 

We found that the best solution was to build dimensions into our metrics so that we could view metrics by team. Once we had metrics cut by team, we could set top-down expectations and prioritize these metrics compared to other work. We were happy to see that individual teams did a great job of taking ownership of and improving their metrics and, consequently, the company’s top-level metrics.

An example: transaction run duration

Coming out of our virtual trade show, Faire Summer Market, we knew we needed significant investment in our database utilization. During the event, site usage pushed our database capacity to its limits and we realized we wouldn’t be able to handle similar events in the future.

In response, we created a metric of how long transactions were open every time our application interacted with the database. Each transaction was attributed to a specific team. We then had a visualization of the hottest areas of our application along with the teams responsible for those areas. We asked each team to set a goal during our planning  process to reduce their database usage by 20% over a three-month period. The aggregate results were staggering. Six months later, before our next event—Faire Winter Market—we experienced a 1.6x multiplier to our incoming traffic, but during the event we were nowhere close to maxing out our database capacity. Now, each team is responsible for monitoring their database utilization and ensuring it doesn’t trend in the wrong direction.

a. Managing metrics with KPI scorecards

We’re moving towards a model where each team maintains a set of key performance indicators (KPIs) that get published as a scorecard reflecting how successful the team is at maintaining its product areas and the parts of the tech stack it owns.

We’re starting with a top-level scorecard for the whole engineering team that tracks our highest-level critical KPIs (e.g., Apdex, database utilization, CI wait time, severe bug escapes, flaky tests). Each team maintains a scorecard with its assigned top-level KPIs as well as domain-specific KPIs. As teams grow and split into sub-teams, the scorecards follow the same path recursively. Engineering leaders managing multiple teams use these scorecards to gauge the relative success of their teams and to better understand where they should be focusing their own time.

Scorecard generation should be as automated and as simple as possible so that it becomes a regular practice. If you build a process that requires a lot of manual effort, you’re likely going to have trouble committing to it on a regular cadence. We strive to have all of our metrics pushed into Snowflake data sets and we visualize everything in Mode reports. Many of our metrics start in DataDog; we use their API to extract relevant metrics and push them into Snowflake so that we aren’t pulling from multiple sources.

As we’ve rolled this process out, we’ve identified criteria for what makes a great engineering KPI:

• Can be measured and has a believable source of truth. If capturing and viewing KPIs is not an easy and repeatable task, it’s bound to stop happening. Invest in the infrastructure to reliably capture KPIs in a format that can be easily queried.
• Clearly ladders up to a top-level business metric. If there isn’t a clear connection to a top-level business metric, you’ll have a hard time convincing stakeholders to take action based on the data. For example, we’ve started tracking pager volume for our critical services: High pager volume contributes to tired and distracted engineers which leads to less code output, which leads to fewer features delivered, which ultimately means less customer value.
• Is independent of other KPIs. When viewing and sharing KPIs, give appropriate relative weight to each one depending on your priorities. If you’re showing two highly correlated KPIs (e.g., cycle time and PR throughput), then you’re not leaving room for something that’s less correlated (e.g., uptime). You might want to capture some correlated KPIs so that you can quickly diagnose a worrying trend, but you should present non-duplicative KPIs when crafting the overall scorecard that you share with stakeholders.
• Is normalized in a meaningful way. Looking at absolute numbers can be misleading in a high-growth environment, which makes it hard to compare performance across teams. For example, we initially tracked growth of overall infrastructure cost. The numbers more than doubled every year, which was concerning. When we later normalized this KPI by the amount of revenue a product was producing, we observed the KPI was flat over time. Now we have a clear KPI of “How much do we spend on infrastructure to generate $1 in revenue.” This resulted in us being comfortable with our rate of spend, whereas previously we were considering staffing a team to address growing infrastructure costs.

We plan to keep investing in this area as we grow. KPIs allow us to work and build with confidence, knowing that we’re focusing on the right problems to continue serving our customers.

4. Keep teams small and independent

When we were a company of 25 employees, we had a single engineering team. Eventually, we split into two teams in order to prioritize multiple areas simultaneously and increase velocity. When you split into multiple teams, things can break because people lose context. To navigate this, we developed a pod structure to ensure that every team was able to operate independently, with all the context and resources they required. 

Today, we’re organized into five pillars, with each pillar containing multiple groups, and each group containing multiple pods. We now have about 30 pods.

When you first create a pod structure, here are some rules of thumb:

• Pods should operate like small startups. Give them a mission, goals, and the resources they need. It’s up to them to figure out the strategy to achieve those goals. Pods at Faire typically do an in-person offsite to brainstorm ideas and come up with a prioritized roadmap and expected business results, which they then present for feedback and approval. 
• Each pod should have no more than 8 to 10 employees. For us, pods generally include 5 to 7 engineers (including an engineering manager), a product manager, a designer, and a data scientist.
• Each pod should have a clear leader. In our case, we have an engineering manager and a product manager co-lead each pod. We designed it this way to give engineering a voice and more ownership in the planning process.
• Expect new pods to share headcount. While this isn’t ideal, there isn’t any other way to do it early on. Resources are constrained, and you need a combination of seasoned employees and new hires on each pod (otherwise they’ll lack context). Pick one or two people who have lots of context to seed the pod, then add new members. When we first did this, pods shared backend engineers, designers, and data analysts, and had their own product manager and frontend engineer. 
• If you only have one product, assign a pod to each large feature. If there’s not an obvious way to split up your product surface area, try to break it out into large features and assign a pod to each. 
• Keep reporting lines and performance management within functional teams. This makes it easier to maintain:
  • (1) Standardized tooling/processes across the engineering team and balanced leadership between functions
  • (2) Standardized career frameworks and performance calibration. We give our managers guidance and tools to make sure this is happening. For example, I have a spreadsheet for every manager that I expect them to update on a monthly basis with a scorecard and brief summary of their direct reports’ performance.

Alternate Example: Caviar/DoorDash

‍Caviar/DoorDash all eventually shifted to variations of “full-stack audience teams”. We outline these below:

• These are teams that are self-sufficient and can achieve all their goals without external headcount (e.g., PM, frontend, backend, designer, and data)
• Teams were organized around four groups: Dasher, Consumer, Restaurant, and Internal (support, ops). These are the key drivers of the business‍
• Below customer facing teams, they had a platform team that served an internal customer → the EPD team‍
• They adopted this model at ~25 people and scaled through DoorDash acquisition. This is also the way DoorDash organized itself going into IPO.

Example
Dasher team (e.g., 100 people). This is further subdivided into smaller teams with a max size of 12. The advantage versus pods is these teams have a single leader, have long-term ownership as they align to the business, and are autonomous. Each of these full-stack orgs was broken down into smaller teams.

Platform team and services based architecture become key for scaling over time

• The platform team's goal was to drive EPD velocity (i.e., reduce blockers, maintain platform speed, internal tooling)
• They scaled this at roughly 10% of the total engineering team. They would build well-designed internal services to ship faster
• In the case of Brex, the platform team was 40% of EPD because they had to invest heavily into payment and money movement services
• You need to have a services-based architecture where teams can plug into internally maintained services for this too work
• When to switch to services-based architecture? Once you hear complaints in EPD. This will happen by 100 engineers.
• Do not switch too early as monolith systems are easier to manage when you are small

How we stay on top of resource allocation: Census and Horsepower

Our engineering priorities change often. We need to be able to move engineers around and create, merge, split or sunset pods. In order to keep track of who is on which team—taking into account where that person is located, their skill set, tenure at the company, and more—we built a tool called Census.

Census is a real-time visualization of our team’s structure. It automatically updates with data from our ATS and HR system. The visual aspect is crucial and makes it easier for leadership to make decisions around resource allocation and pod changes as priorities shift. Alongside Census, we also built an algorithm to evaluate the “horsepower” of a pod. If horsepower is showing up as yellow or red, that pod either needs more senior engineers, has a disproportionate number of new employees, or both.

‍

‍

Pods are colored either green, yellow, or red depending on their horsepower.

II. Building an engineering recruiting brand

By Tido Carriero, Co-Founder of Koala, Ex-VP of Engineering/CTO and CPDO at Segment

Hiring for engineering is 10x harder than any other function in a young startup. Oftentimes you are competing for the best engineers amongst top tech companies and you have to start thinking of creative ways to set yourself apart from the competition. 

Having a strong engineering brand means that a company is known for working on interesting engineering problems and has been able to attract the best engineers as a result. Over time, this will pay dividends: More qualified inbound candidates will enter the pipeline on their own, your outbound success rate will increase, and your close rate will increase.

There isn’t a one-size-fits-all model on building a great engineering brand. What works for one company might not work for yours. We spent time with Tido Carriero, EX-VP of Engineering and CPDO at Segment to outline strategies for building engineering brand.

1. Why is engineering brand important and when should you start investing in it?

Your engineering brand is a subset of your employer brand. Both are different from your overall company brand, which is more about your product and is directed at customers. Your employer brand summarizes what makes your company a great place to work. Your engineering brand is about (1) the hard problems you’re solving; (2) the caliber of your engineering team, particularly your senior engineers (e.g., Dropbox was able to hire many engineers in the early days because they had Guido van Rossum on the team, the inventor of Python); and (3) the business success that will come as a result of solving hard engineering problems.

It often takes a lot of effort to build an engineering brand, so don’t treat it lightly. As a founder, you’re constantly triaging problems, weighing the tradeoffs and benefits of solving each. Think of building an engineering brand as a solution to a problem, and decide when to invest in it based on (a) how painful that problem is for your startup (i.e., how much is engineering hiring progress getting in the way of growth?) and (b) whether you’ve explored other less expensive options (i.e., are there things you could be doing to improve hiring funnel conversion?).

Typically, the breaking point is when you’re spending over 40+ hours of engineering time on acquiring each engineer. If your team is complaining about constantly interviewing and never having enough engineers to keep up with scale and technical debt, this may be a sign to start investing in your engineering brand. 

“In our second year we were spending over 80 hours per hire, mostly because I cared so much about the quality of people joining us and we didn't have a strong brand yet. In 2023, we’re spending less than 23 hours per hire.” — Marcelo Cortes, Faire

Once you have this sign, the first place to look is your close rate. This is because close rate is the biggest indication of the number of engineering interview hours spent per hire. For example, if your close rate is 45% instead of 90%, you’re spending at least 2x the number of engineering hours on hiring as you could be. A close rate of 90%+ is world class, 60–80% is OK, and below 60% is where you have a problem. Maybe you’re bringing in too many candidates whose comp expectations are wildly misaligned with what you can offer. Spend some time and attention diagnosing why candidates are declining and see if you can make tweaks higher up in the funnel.

Other common engineering hiring problems are:

• Number of engineering hours spent in interviews is high

• Best in class is 20–25 hours from start to close (including people you don’t move forward with). If you’re closer to 50 hours, your engineers are likely feeling the strain. 100 is unacceptable. A good metric is: ‍
  • (number of engineering hours every quarter – number of interview hours from Greenhouse) / number of offers accepted‍
  • Even if your interviews are 30 minutes, assume one hour per interview to account for writing up notes and attending debriefs.

• Not enough top-of-funnel volume
• Do things to drive up referrals: Spend more time sourcing, hire a sourcer, source from different channels, etc.
• Not having clarity on what you're looking for / realistic expectations of the market
• Force hiring managers to outline only three must-haves and a more specific job rubric.
• Compensation problems / targeting the wrong ICP (e.g., candidates who are ultimately only interested in FAANG-level comp)
• Company/role pitch problems

All of the points above feed into your employer brand. If you’ve exhausted these avenues and something still feels broken, it’s likely time to start thinking about your engineering brand.

“Consumer companies with early success have a huge advantage vs. B2B companies like Faire. We started building an employer brand when we started to hire faster, about a year in. We started by creating our blog and publishing our engineering handbook. It was very cool to see candidates mentioning that they got interested in Faire after reading the handbook.” — Marcelo Cortes, Faire

For another company (that later became world class at hiring engineers), the trigger was:

“We were supposed to be hiring five people a quarter and were getting two or three at the quality that we wanted. There were several quarters of missing targets. Our close rate was really bad, 30 or 40%. We said, ‘We need to do something different. Our head of engineering working 80 hours a week, 75% of it on recruiting, and still missing the numbers is not the long-term solution.’ We were consistently missing targets despite lots of effort. We were like, ‘We need to invest in something more scalable.’”

One more note: Investing in an engineering brand often entails having to make difficult tradeoffs on other priorities, so it should be a decision that the entire leadership team—and even the board—is aligned on. Everyone should understand and be on board with the possibility of other projects slowing down.

2. How do you get started?

Think of your engineering brand like you would your marketing engine. If you have 20 engineers on your team, you’ve already converted 20 customers. Your task is to figure out their “customer journey” and recreate that at a greater scale. 

Start by doing a simple survey of your engineers. If you have 10 or fewer, you might even interview them in person. Or survey the entire team and interview your best five or 10 in depth. Ask them:

1. How did you first hear about our company?
2. What played the biggest role in your decision to accept the offer?
3. Is there anything else about our engineering work or the team/culture that you think we should be sharing with the world?

Specifically, the outputs from this process should be the (a) messaging and (b) channels that are already organically working. At Segment, for example, Calvin French-Owen was a natural storyteller and had already been writing blog posts on engineering problems he found interesting. Both open source and Calvin’s blog posts were leading to the same volume of candidates but the survey findings revealed that it was actually Calvin’s posts that was attracting candidates with better long-term fit.

Faire surveyed their team of ~25 engineers to uncover the company’s most effective selling points:

1. Hyper growth 
2. Modern infrastructure and programming languages (Kotlin, React, etc.)
3. Working with the founders who built Square Cash
4. Clear career ladders and opportunities to grow
5. Company mission (hiring entrepreneurs to help entrepreneurs)
6. Internal practices and processes focused on efficiency
7. Competitive compensation (If you want the best, you have to pay for it. Faire’s founders spent a lot of time selling candidates on the value of the company’s stock options, often using Square as a reference for potential future growth.)

3. What works well on each channel?

a. Blog posts and Hacker News

Hacker News is an incredible tool to build an engineering brand. Since making it onto the front page of HN can be tough, here is guidance on what works well:

• Get to know the community. The best way to get a sense of what resonates and what doesn’t is to put in the time and immerse yourself in HN. Spend a few days reading and commenting on threads. Talk with employees who are HN readers. Consider hiring popular HN members as a way to build stronger bonds with the community. For example, Key Values founder Lynne Tye has developed a good reputation within the community.
• Treat readers as intellectual peers, not people you’re marketing to. Readers come to HN to have their curiosity satisfied. They are hacker types—enterprising, technical, curious. Assume high technical knowledge and avoid marketing speak. If your goal is to hire engineers, you want to signal that working on your team means working with smart people on interesting problems.‍
• Ask your more enterprising engineers to try drafting something. Give them the prompt: “If I’m a bored reader on the internet, what will hook me enough to read the whole thing?” or “What’s something that you wish you’d known six months ago that would be useful to the project you’re working on now?” Or spend an hour “interviewing” an engineer: Ask them how a piece of infrastructure works or to explain the project they’re working on as they would to one of their smart friends. It often helps to pair engineers with writers to help them carry the story through. You can also try compressed experiments, like a blogging week.‍
• Find the best versions of the headline and first paragraph. You have only a few seconds—about two paragraphs—to get the reader’s attention before they jump to another page. Rewrite your intro until you feel confident you’ve got it. Introduce yourself and the problem you’re trying to solve, pretending that your reader has no knowledge of or affinity with your company. Keep it simple but compelling: “Foo is an X that does Y. Our users are Z. Because of that, we have to deal with W.” A/B test different headlines and paragraphs.‍
• Give a vivid, thorough explanation of the problem. You want your problem to be hard enough that it isn't obvious how you would solve it, but it's obvious why you'd want to. Communicate three things: (1) what it is, (2) why it matters (what pain it causes), and (3) what makes it hard to solve (mentioning failed attempts is great). Or if your post is about frivolous (but interesting) hackery—like making a Turing machine out of Lego—you can say you wanted to do it “just because.”‍
• Tell a story, don’t give a tutorial. Tutorials optimize for quick answers; stories incite curiosity. Talk about the complex path you went down before you could make things simple. “We needed to do X so we tried Y, had trouble because of Z, solved it with W, and there were some surprising details A, B, and C.” If your personal experience is relevant, use it—it adds to the story and shows people that you’re a fellow human, not a company. Here’s a good example from Segment.‍
• Use factual language, not marketing language. Readers will feel intellectually disrespected and turn on you. Don't address other users by their usernames. Don't introduce yourself more than once. Don’t use high-level bullet points; unlike customers, HN readers want to get into the details of the problem rather than just be presented with a tidy solution.‍
• Continue engaging in the comments. Add a first comment where you give additional backstory or mention areas you'd like feedback in. If readers ask questions, give genuine, straightforward answers. It’s good to be humble or even mildly self-critical. A little humor and enthusiasm is ok, but don’t overdo it. Try to avoid insider discussion with friends, batch mates, etc. Don’t ask for upvotes, and don't ask your employees/friends to post booster comments or softball questions.

Cheat sheet: How to write blog posts for Hacker News

Don't do this

• Talk to customers/users
• Speak like (or on behalf of) a company
• Give a tutorial
• Say nice things about yourself or your product
• Assume people know who you are and what you do
• Glaze over the problem
• Easy, straight-line solutions
• Be sensational or indignant

Do this instead

• Talk to fellow engineers/entrepreneurs
• Speak like a person
• Tell a story (“we needed to do X so we tried Y, had trouble because of Z, solved it with W, and oh by the way have some surprising details A, B, and C”)
• Talk about challenges you’re facing
• Give a refresher on who you are and why your problem is interesting
• Make the problem interesting enough that readers are “fueled up” to hear the solution
• Hard problems, false starts, unexpected turns, complexities, other behind-the-scenes info
• Intellectually interesting

Segment's Experience with Hacker News

Segment produced its engineering blog with the specific goal of getting on the front page of HN. The company pairs its engineers with writers to produce one post a month and tries to get as much feedback as possible. Only 20–30% of posts turn into a big hit, but they’ve found the investment to be worth it. Posts might give a behind-the-scenes look at how Segment’s engineers solved a particular problem or tap into the community’s intellectual curiosity with a historical deep dive into a certain technical standard.

• Don’t ask your entire engineering team to write content. Good storytelling is a specialized skill. At Segment, only a handful of engineers regularly write.
• Each post takes almost a week of work. At Segment, each post takes 30–40 hours of a talented engineer’s time. A big tradeoff, but it’s paid off for them.
• Get as much feedback as possible before you post. In fact, just write the headline and first paragraph and get feedback on that. Do different versions and choose the winning one.
• Choose the right metrics. Segment’s objective is to get on front page of HN. They track the number of pieces they publish each quarter, the reach generated by those pieces, and the share of job candidates who came in for interviews because of the blog.

Stripe's Experience with Hacker News

Stripe introduced itself to the world via a post on HN and has since had a wealth of success with HN posts and product launches. Stripe’s content includes highly technical blog posts like this one on rate limiters, Atlas guides on how to run a business (The business of SaaS, Pricing low-touch SaaS), and posts on the company’s inner workings (Stripe Home, BYOT: Bring your own team).

• Manage the process with content councils and DRIs. Blog posts are coordinated by informal “content councils,” typically consisting of the engineer who had the original idea and members of its content and developer relations teams. The post is assigned to a DRI (directly responsible individual) who sees it through to publication. It takes about two weeks to produce a post.
• Do external “peer reviews.” Stripe sends drafts to engineers in its network for feedback.
• Involve the founders. When Stripe launches a feature or product on Hacker News, either John or Patrick will block time to answer questions and address feedback.

b. Communities

Community building comes in different forms. It might include speaking at conferences, cultivating a community online, hosting events, or building an on-campus presence. 

Early on, Dropbox wanted to attract engineers from top computer science schools, so they built an intern program which included mentorship, being able to work on hard technical problems, free food and housing, and the potential for a full-time position at the end of the summer. The founders were MIT alumni and used their ties to the school to attract top students. Every fall, they would attend career fairs, host tech talks, conduct interviews on campus, and hire teaching assistants of the hardest CS classes. Over time, Dropbox built a university recruiting machine where students flocked to their internship and new grad program.

A great way to start building your brand is through in-person events at your office. Both Segment and Brex hosted events that attracted software engineers. Segment famously held “Segfault” every quarter. The important thing here is not to brand it as a recruiting event. The words “we are hiring” or “come apply” should never be mentioned. These events should not be too content-heavy and it doesn’t need to be very structured. The important thing is coming up with a topic that attendees will find interesting. The networking aspect is what attendees will find most valuable.

For example, Segment asked their engineers for the names of three to five people they’d love to work with:

“We'd say: ‘This is not a recruiting event. It's not going to feel like they're getting pitched Segment. This is really cool content, free beer, free pizza, free cupcakes; it's just going to be a fun night. It's a half-hour mixer, 45 minutes of content. People are welcome to stay. People are welcome to leave. Super casual, invite your friends.’ Everyone understood it was a recruiting event internally, but because it didn't feel like a recruiting event, people were comfortable inviting their favorite former coworkers. If I had said it was a recruiting event and we were advertising Segment, the invite conversion rate would have been either lower or a less prestigious group.” — Tido Carriero, former CTO

The main draw of these events is to not only build a community, but to start building your brand. Even if an attendee isn’t looking for a new job today, they might be nine or 12 months from now, and when that day comes they’ll likely remember you from the event. 

c. Open source

Using an open source platform such as GitHub is another way to start building an engineering brand. This is a big time commitment and easier to do as you scale and have resources to focus on it. Young startups that invest in open source tend to be the ones building open source companies. You also need internal expertise to do it well. For example, Jesse Wilson and Jake Wharton managed some of the best-known open source projects and their expertise was helpful to build an engineering brand through open source at Square. 

Companies like Segment and LinkedIn posted projects on GitHub early on, allowing engineers to contribute to the repository. The purpose was to create awareness around the technologies that were being used, showcase projects in development, and most importantly allow engineers to contribute to a project and get a sense of how things worked at this company. Both companies were very intentional about making sure they were putting out interesting projects.

In turn, engineers who have already had a chance to contribute to a repo almost have an advantage during the interview process because they’re already familiar with the technologies being used. Although open source isn’t a recruiting tactic, it often can result in a hire during brand awareness.

III. Building an engineering recruiting machine

By Cosmin Nicolaescu, CTO at Brex

By now you’ve been able to build a solid engineering team of 15+ engineers and you need to start thinking about how you’re going to double. It’s likely that your current recruiting practices are not going to scale and you need to start thinking about ways to be more efficient without compromising on the quality of talent.

We recommend that you start thinking about building a more efficient recruiting process for engineering when you’re looking to add more than five engineers per quarter and b) time to hire per engineer is over 30 days. At this point, it’s also worth bringing on a head of engineering whose job for the first year is to build an engineering org. 

When scaling engineering recruiting, the main components to get right are:

1. Measure and be able to predict funnel conversion
2. Standardize your process and train interviewers
3. Make recruiting part of everyone’s job
4. Recruiting team

In this post, we draw upon lessons from Brex. We spoke with their CTO, Cosmin Nicolaescu, who joined the company at 40 employees and has continued to scale it to over 1,200 today, with most of his impact being on engineering. 

1. Measure and be able to predict funnel conversion

One of the mistakes we often see early on in engineering recruiting is that founders aren’t keeping track of—and therefore aren’t diagnosing and improving—their conversion funnel.

“We were passing 80%+ of people and converting less than 10% onsite. That screening is not efficient. You might as well get rid of it and save the two hours because you would probably end up with the same result.”

To understand conversion, work backwards from how many hires you want to make this year. From there, you can get a sense of how many hires you need to make per month to meet this goal. Given this information, are you currently meeting your hiring goals? If the answer is no, it’s time to debug your conversion funnel.

The most common problems are:

• Your offer acceptance rate is very high, but overall candidate volume is low.
• Your top-of-funnel conversion is high, but your offer acceptance rate is very low.
• It takes over 30 days to hire one engineer.

High offer acceptance, not enough candidates

We often hear founders say that they have an offer acceptance rate of 80–90%, yet they’re still not meeting their monthly hiring goals. An 80–90% offer acceptance rate is great; the industry average is closer to 50%. This means that you’re able to get candidates excited about your company and you’re calibrated on the types of people you’re looking for. But if you’re missing your hiring goals, it’s likely that you have a volume problem. 

At this stage, most of your engineering candidates are likely coming either from internal referrals or they’ve been passively sourced. A general rule of thumb is for a hiring manager to do five to six interviews a week. Individual contributors should expect to do three to four technical interviews per week. If you’re not seeing these numbers, it’s a good idea to either (a) hire a dedicated sourcer (either contract or full-time) to increase top of funnel volume or (b) set a goal for your recruiters to source 100–200 candidates a week on top of their current workload (or both).

Low offer acceptance, healthy top of funnel

Another common problem is founders complaining that their offer acceptance rate is abysmal. They’re seeing a healthy number of candidates through the pipeline and many are even making it to the offer stage, but eight out of 10 decline their offers. In this scenario, it’s likely that there is miscalibration at the top of the funnel. 

“The original process we had resulted in good hires but really bad conversion rates. All interviews were done by four people, including the two founders. They had two phone screens, none of which covered technical things; they were just conversations. The pass rate for phone screens was high. Then they had one four-hour interview that was split into two sessions. One was designing something and implementing it, and most people would fail. That had a 10% conversion rate or something. It wasn't scalable. You couldn't get more people to do it without quality dropping.” 

A few things to consider when screening candidates:

• Motivation: Why is the candidate interviewing with your company? If someone is interviewing at FAANG and your startup, really dig deep into why they’re excited about you. A lot of candidates know how to interview and will have canned responses. Make sure you and your recruiter are digging into their motivations early on.
• Compensation: Did you communicate your compensation philosophy early enough in the process? Did you discuss it with the candidate before getting to the offer stage? By the time a candidate receives an offer, they should not be surprised by the numbers.
• Timing: Do you consistently lose candidates because they’ve decided on another opportunity by the time you’ve extended the offer?

“If the close rate went under 80%, we would look into decline reasons. We do it by pipeline. We have backend, frontend, mobile. Every month, we look at a report that breaks down by role, location, level, gender, to see if there's different patterns. Looking at just the overall rate can mask problems. So we break it down and look at every pipeline.”

‍How Brex improved its conversion funnel:

Old process

1. Initial screen (conversational, not technical)
2. Four-hour technical interview (conversational)
3. Decision

New process

1. Initial screen (why they’re leaving, ideal role, etc.)
2. Technical screen that actually involves coding
3. Implementation test in an actual code editor
4. Debugging and testing an existing problem
5. Structured values interview
6. Internal debrief
7. Hiring committee approval

Time to hire is over 30 days

When you need to hire many engineers at once, time to hire becomes an important metric to track. You want to optimize for speed without compromising the quality of the hire. There are a few things that can slow down a hiring process:

• The founder meets with every candidate and is the bottleneck.
• There aren’t enough employees interviewing, which leads to delays in scheduling.
• The decision to hire takes more than two days.

Once you’ve identified where you’re falling off with candidates, you should work on improving each stage of the funnel to create a more efficient process that scales. 

The best startups are able to move candidates through much faster than 30 days. Even at 1,200+ employees, Brex still moves as fast as needed to close candidates, and can move from first conversation to start date in less than a week. 

The key takeaway here is to ensure that you (the company) aren't the bottleneck, and you can move as fast as possible and be flexible with candidates. That means the recruiting team is on top of the candidate process, and recruiting coordinators have good tools and processes so they can quickly book interviews, and schedule debrief sessions. On top of that, having a streamlined offer approval process where you can quickly churn out an offer once a decision to hire has been made will reduce the time to hire. 

2. Standardize your process and train interviewers

A common challenge at startups is that they lack experienced interviewers. As a result, they don’t know how to balance tradeoffs and instead look for unicorn hires. Training your interviewers and instituting a standardized process will result in faster, more efficient, and more accurate hiring. In addition to training everyone who is doing interviews, you should also take steps to make sure everyone is calibrated on what an excellent hire looks like for your company. 

“In order to interview at Brex, you go through an onboarding process. You shadow twice and reverse shadow twice. The number one thing is: Do we feel you're calibrated? As you meet people, will you come back with the same result? The way calibration works is: Two of us are in the same interview, we both take notes. I'm driving, you're observing. You enter your feedback, I enter mine, and we compare notes. Whenever you have a disconnect, have a conversation and do it again until you get better calibration. We did two shadows and three reverse shadows initially.”

“We emphasize written feedback. I would go through every candidate and if people didn't have raw notes with a ton of details, I would give them feedback: ‘You said this person meets our values, but you didn't say why. I want to see the raw notes. I don't care about your interpretation as much. I want to see the raw notes that led you to that conclusion, otherwise there's zero calibration.’”

Interview committees are sometimes put in place for calibration purposes. If you have the resources to do this, we highly recommend you put this in place to keep a consistent level of caliber. Be careful not to let it slow down the process though. Create a system where people can give feedback quickly and easily. At Brex, the committee consists of Cos or one of his direct reports as well as their head of technical recruiting.

“The committee is async. You get a response within less than an hour, because there's always someone that can review the packet quickly. The debrief is done within one day. If you do interviews today, you have the debrief tomorrow so that it's fresh in the interviewer's mind. Once the debrief is done, the recruiter posts for approval, and approval is a couple hours at most. If a candidate is in a rush, the recruiter flags it and we expedite. We will always move as fast as needed.”

3. Make recruiting part of everyone’s job

If you want to double from around 20 to 40 engineers in the next year and only 10% of your team are interviewers, this is not going to scale. Not only do you run the risk of burning out your best interviewers and likely best performers, but you are putting yourself at risk of losing top candidates due to timing.

We highly recommend having all of your engineers conduct interviews or be actively involved in recruiting in other ways. You can think of your company as a ship: Every person has a role to play in order to keep the ship afloat and moving towards the destination. Set the expectation early on that everyone should be involved in recruiting, whether that’s interviewing, sourcing and referring, doing sell calls, etc.

“If you’re opposed to doing interviews, you have to contribute towards recruiting in a different way. You have to do events, you have to source, you have to do something to make up for it. Everybody needs to be involved.” 

At Brex, IC engineers are expected to commit to two hours of interviews per week. Managers are expected to do four hours. Directors are expected to do six. The best interviewers are celebrated at all-hands. 

You should also be regularly generating referrals from your team. The critical things to get right here are (1) to make it an easy, positive experience for employees and (2) infuse it into the culture by celebrating it at all-hands and other forums. This is far more important than offering a financial incentive. 

“We tried to create both the culture and the systems to make it easier for people to be involved. On top of that we had the recognition aspect and celebration to make sure that people get praise for those things. We made it really easy to do a referral. In all-hands we would talk about people who had the most referrals. We didn't do the bonus because we felt that you already have incentive. If you have someone that you really enjoy working with, $10,000 is not going to make a difference. Wanting to work with them is a more valuable incentive.”

“The biggest fear that employees have with referrals is, ‘I’m going to recommend my friend and they're going to have a shitty experience.’ We make sure that they get fast processing and a high-quality process. We’re extra careful and give them white glove treatment. We also involve the employee in the final decision. For every referral, before we tell the candidate, we tell the employee the decision. If we don’t move forward with someone, we give them a heads up so they're not surprised.”

4. Recruiting team

Building a strong engineering recruiting machine won’t be possible if you don’t have a strong technical recruiter. More so than ever, you need recruiters who are sharp, efficient, and are thought partners. Founders often get this wrong: More recruiters do not always translate to more hires. If you’re able to hire a recruiter that is efficient, they can typically do the job of two or sometimes three recruiters. 

The role that a recruiter plays in building an engineering recruiting machine is:

• They’re extremely calibrated on your engineering profile.
• They have a closing strategy for every candidate.
• They are able to source high-caliber candidates and get them through the door.
• They know how to probe candidates and don’t treat the recruiter screen as just a formality. 
• They know when and how to give you feedback on the interview process and ways to improve.

Having a strong recruiter is not the magic answer to all your recruiting problems. As the founder (or hiring manager), you need to be just as invested as the recruiter in hiring engineers. Work closely with your recruiter to hone their company pitch. You should feel confident that anyone that makes it past their phone interview will pass the cultural and sometimes even technical aspects of the interview process. You want a recruiter that is as picky with candidates as you are and can give you a detailed profile of any candidate that has a conversation with you.