Pilot Program Metrics for Early Stage Startup Success

What key metrics does your pilot program need?

by Lenka Davis

Startups generally consist of teams that have not worked together for very long. In order for everyone to be as prepared as possible for a new product roll out, it’s important to do what you can to set them up to succeed. Leading your team from small projects all the way through bigger and more complex projects, is one way to make sure everyone is ready. Pilot projects are good programs to run because they give the team the ability to learn and improve when the stakes are not so high.

Pilot program metrics will let the team know what the goal is.

Once you have designed your product or service, and built the MVP (minimum viable product), then it’s time to run a pilot program. A pilot program will test the viability of your minimum viable product and give you insight into how your customers will benefit from it, and then what features you might need to build next. Definition of pilot program.

“At its core, a pilot’s purpose is to evaluate, uncover knowledge and useful insights, and result in either design improvements or alternative solutions.”

— method.com

Monday.com describes a B2B pilot project example as follows:

“B2B SaaS: Testing a new product or service in a small market or with a limited group of customers before a full-scale launch.”

— monday.com

How do you know that your pilot program was successful?

Does each team member know what the purpose of your pilot program is?

Does your team have the knowledge on how to give your customer an A+ experience?

The answers to these questions will be in the metrics set up for the pilot program. Read the example below on how we set up a client to be read for their pilot program.

The project

Our client was interested in finding a customer willing to do a paid pilot program. The project was going to be paid rather than done at no cost. Since it was in the B2B space it would have a long schedule, the need to set up a customized software module with the potential to save the client a significant amount of money. A paid pilot program therefore was the best option.

To get the whole team heading in the same direction, we decided to document the pilot program metrics. As the team started discussing possible metrics it became clear that, depending on the department you were representing, you had slightly different metrics you wanted to measure. In a startup, team members wear many different hats, and so they will have different input into what the metrics could be.

We set up the metrics around the following areas:

Reliability 🚙

Security 🔐

Satisfaction 😊

ROI  💵

Conversion 🤝

Message ✏️ 

Timeliness ⏳

Reliability 🚙 and Security 🔐

These areas mostly focused on the internal processing side of the business. Can the information and data be moved around securely and once the process was completed was the business able to be confident that the reports were reliable.

Timeliness ⏳ and ROI 💵

These areas mostly focus on the customer experience and how they rate it. Did the business surprise and delight customers? The customer needs to have some return on investment (ROI) in order for them to either recommend this service or continue to use it in the future. Part of the satisfaction is the information delivered in a timely manner, based on what the customer expected.

Satisfaction 😊 and Conversion 🤝

Once the pilot project is completed, what was the satisfaction rate based on either a survey or feedback. Often when teams engage in pilot projects they understand that not everything will go smoothly but they are willing to overlook this since they believe the end result is well worth the pain or wait. If the customer signs up for another project, and converts to a paying customer, or recommends the product to someone else then that is regarded as a good measure of satisfaction.

Message ✏️

Depending on the relationship the team has with the pilot customer, collecting feedback is critical. Feedback can then be used when creating messaging to future customers. It can be through an official survey or just discussions. The terms the pilot customer uses, how they speak about the product and how they describe their experience with the product is most likely how other prospects just like them will think about the solution. If you have existing messaging, check if it resonates with the customer. 

For each of the above groupings, set up measurable metrics that the team can agree to. This can be done through a survey, or any numeric measure. Numeric measures can include the tracking of users or deliverables or a percentage increase in a volume or speed. Using the SMART methodology to establish your metrics will help them be clear and easy to understand.

Fun Fact

Sometimes pilot programs result in unexpected results. One famous example is when 3M scientists were trying to develop a super-strong adhesive and then by accident found a reusable adhesive instead. The birth of Post-It stickies still took a few years to make it to the market. Another 3M scientist was frustrated during church choir because the little scraps of paper marking the hymns would fall out in a few days. Then he remembered a seminar he attended at work given by a 3M colleague that showed a removable adhesive. The two scientists talked and came up with the idea. “I thought, what we have here isn't just a bookmark," said Fry. "It's a whole new way to communicate." A need for a bookmark that would stick to paper without damaging the pages was turned into a product and then was pilot tested as sticky notes in Boise, Idaho with 90% of those who tried it said they’d buy it again!