top of page

Pilots. Not the kind in the sky.

In the software development world, failing fast is common thing. The Lean Startup is a way of life. My mentors often remind me to get our products out in the hands of users quickly to learn. In a heavy engineering or manufacturing environment, the culture does not necessarily support this thinking. In manufacturing, adherence to a design is critical, so experimenting or trying out new things with a seemingly haphazard approach may be a problem. Merging the tech startup world with manufacturing, which is necessary for Industry 4.0 adoption can be tricky. There is a way to implement something in a structured way through a pilot.

A pilot is a small scoped experiment to test the value, feasibility, cost of a project before scaling. The goal is to focus on learning as much as possible with minimal operational impact. When my team implemented a new module within our quality system, a common best practice we used was creating a pilot. Over the last 10 years, I've seen both full scaled launches, as well as a pilot approach. It feels like the feedback received is about the same, however the impact is much less when the scope is reduced.

There are 5 basic phases of a pilot:

  1. Scope the pilot. I recommend using the project charter template, as described here: The team involved in the pilot and the location should be a small subset of the larger project vision, if the project is a planned implementation. Pilots can also be used to experiment with new technology or to validate a hypothesis that a solution will solve a problem. In those cases, the project charter can stand alone. One last thing to consider is the duration of the pilot. Having a defined end date can help settle the minds of the people who are not yet early adopters.

  2. Identify the learning metrics. A pilot project offers a low cost opportunity to learn during hypothesis testing and spending time figuring out to measure the test is important. For example, if the hypothesis is that the shop floor operator will be able to enter the data into the system easily, then the measurement could be the hours of training required to learn the system. Another measure could be the quantity of feedback received from the shop floor personnel. A second hypothesis could be around operational cost of data collection. The metric could be the cost of the cloud computing and storage.

  3. Plan and execute the onboarding. I have seen the word onboarding and I cannot think of a more appropriate term. Onboarding encompasses the training of the team who is involved in the pilot, as well as the defined processes for feedback. For more information on a robust feedback process, check out this blog post: Since this is a pilot with a defined duration, establish a cadence around feedback and iterations up front. Roll out the process and training in advance of the official pilot launch date.

  4. Run the pilot project for the defined duration. As a project manager, check in with the people involved in the pilot frequently, even if there is a feedback system in place. Assume silence is not a good thing, after all the team is essentially the customer. If possible in the pilot, iterate through the implementation taking into consideration the feedback captured from the end users. If the pilot is a locked system, such as an off the shelf solution, document the findings and learnings thoroughly. Video is a great way to record this data quickly. The learnings will be used to evaluate, which is the final phase.

  5. Evaluate the learning metrics. At the conclusion of the pilot project, it is important to revisit the learning metrics and record the data for each. At this point the working team should have an internal review and prepare a decision matrix to determine the next steps. Even if the leadership team has the final signoff any scaling decisions, preparing them with a summary of the learnings in an easy way will help their decision.

In summary, a well scoped pilot project in manufacturing will bridge the gap between the manufacturing culture and the Lean Startup way of thinking. Numbers make sense to many technical leaders, so using data collected during the learning process will speed up the decision to continue investing, pivoting slightly, or killing the project altogether. Keep in mind that any of these decisions is correct, when supported by data. Killing a project that is not a good fit for your industry at the minimal cost of a pilot program, is much better than investing in a large scale implementation and reach the same conclusion. Finally, using artifacts such as project charters, schedules and metrics will merge the familiar with the uncertain, and in turn make the pilot more successful.

Here are the phases of a pilot project:

-scope the pilot: people, duration, location

-identify the learning metrics

-plan the onboarding - crucial to have the team prepared with training, as well as how to support

-get feedback often


-evaluate learning metrics

Recent Posts

See All


bottom of page