Build a Pilot Program: Test the Effectiveness of Your Continuous Performance Management Design
A pilot program allows you to learn on the fly as you test out the design elements of your continuous performance management program.
Continuous Performance Management Step #4: Learn on the Fly
You are on a journey… a journey to design and implement a continuous performance management process for your organization. You've come a long way. So far, you've:
Designed a basic program, fleshing out your initial ideas and setting guiding principles
Aligned with your stakeholders, developing the start of a business case to use throughout the project
Discovered what your customers need from a performance management process and developed a value proposition statement
You've also crafted your thoughts into a "final, final draft" that you are ready to test. It's time to celebrate your wins thus far. *pause here for the happy dance*
While we pause (and maybe - definitely - dance), let's get oriented. We are on Step 4 - Pilot of an eight-step process to design and implement continuous performance management.
I am passionate about performance management and was so pleased that it was the most requested topic in the 2021 content survey. I believe a solid performance management process that features frequent check-ins between an employee and manager can be a game changer for business results and employee engagement. After all, a 2016 report from CEB indicates that 95% of managers are not happy with their companies’ performance management process and 56% of employees share that they don’t receive feedback on how to improve their performance. Surely we can get a handle on this, providing a better process for managers and employees.
Pilot Testing
Ok now. You've celebrated Design, Alignment and Discovery. Now it's time to launch a pilot test to see how your initial idea fares.
Hopefully the design you've developed is sound and you're making tweaks during the pilot. Congratulations! If you're finding that the feedback is more than tweaks, don't despair. It's perfectly normal and ok! That's why we engage pilot programs. It gives us permission to test out an idea and go back to the drawing board if it's not just right. Don't give up… it's all part of the process.
“I have not failed. I've just found 10,000 ways that won't work.” ― Thomas A. Edison
What is a pilot, exactly?
So glad you asked. According to ProjectManager.com, "a pilot study is a small-scale project that is used to test the feasibility, duration, cost and risks of the full-scale project. Through this study, improvements can be made to better the performance of the larger project."
A pilot enables you to test and tweak, test and tweak, test and tweak, and so forth. Just like a large tech project may run a small pilot to test out the feasibility of new software, so are we running a pilot to test out the theories behind our design and iron out the wrinkles throughout the rollout. We want to find out if the design actually supports employee - manager discussion on performance. Does it help employees correct performance issues before they become trends? Do participants understand and use the tools and training you've developed to support the rollout. What about the actual performance discussion form? Does it effectively capture the data you need? You'll be testing out every component of your plan - every detail. You'll discover updates needed and likely find items you need to add into your plan.
Pilot testing can be fun. It's busy, sure. It's also a time of learning on-the-go and agilely adjusting your plan.
To launch an effective pilot, you'll be working through three actions:
Build your plan
Select & prepare your pilot group
Gather participant feedback
Also worth noting, I've adjusted plans on the go in the middle of a pilot when it was clear the design wasn't meeting the mark, and then re-launched the pilot with new design elements. You'll make that call depending on what you are piloting, such as testing variations to a single component in the design or piloting a first go at the whole enchilada. For enchilada platters, I recommend allowing lenty of time for participants to get used to the plan before changing up elements. Once you've settled on the main design components that are working, you might re-pilot a few elements to see what works best for your customers. The point being, you will want to be agile in how you apply pilot testing. It's an excellent tool to master for your talent development toolbelt.
First things first, let's plan.
Build a Smashing Pilot Plan
Your plan should outline three important details:
1. Duration - How long will you run the pilot?
The most important consideration to project duration for continuous performance management (CPM) pilot programs is to include enough time for participants to learn, use and get comfortable with the new approach. As with any new program, it takes time for your customers to get the hang of things which then allows them to offer balanced feedback.
For CPM pilots I recommend a duration of one year with quarterly milestones to assess effectiveness and acceptance. Because we want performance management to be something employees talk with their leaders about frequently, it will take time to embed the practice into the day-to-day, set new expectations for the timing of discussions and build performance management muscles. I like quick progress too, but give it the time it needs to really make an impact and learn what you need to learn about how it's working.
If you have done an effective job of garnering stakeholder support, once they begin seeing incremental improvements throughout the pilot, you will have other customers clamoring to participate. That's an awesome thing. But hold your ground and stick with your plan. You can include a selection of those interested customers in the next pilot phase.
I have not tried using quick burst pilots to assess the overall program for the above reasons. I have used more frequent changes to weigh the effectiveness of a particular template or training. It's important to know if there are barriers to using a particular template or training, for example. You'll mark those milestone questions down in your timeline, which takes us to the next plan detail to design.
2. Timeline - What are your plan specifics?
You know you love a good plan. If you don't you should! Setting feasible timelines is key. Once you've set the duration, lay it out on paper / whiteboard / somewhere you are comfortable brainstorming dates and pilot steps. Begin plugging in milestone dates such as:
Deadline dates for performance check-ins throughout the year
Annual performance roll-up rating deadline, if you're testing that
Performance calibration timing for merit decisions
Surveys and focus groups (we'll talk more about those below)
Stakeholder updates
Communications you want to share along the way
Program Input Team updates
If you like a Gantt chart, great, lay it out. Prefer PowerPoint? Great, lay it out with a crisp design. Comfortable in Excel? Fab. Record it there. Lay out the timeline, also tracking owners and actions needed to meet the milestone, so that you can track it like mad, staying on course and attending to the details.
3. Participant Group Size - How many will you include in the test?
A pilot is by design a small segment of your overall population. Your program might be picking up speed and getting popular. Don't be tempted! Focus on an appropriate pilot size.
I recommend keeping it small for the first pilot. You need enough folks to gather meaningful feedback, but not so many that you're managing too many expectations and juggling the administration that comes with a pilot program.
What size will get you a solid 10% representation of your total population? Shoot for that. Even better if you can include a whole department that will represent a majority of the roles you need to support with performance management.
Participant Selection is Key
In the last section we began the conversation about the size of your participant group. Let's shift the discussion to how to select a pilot group and ensure they are well prepared.
I mentioned including a whole department in your pilot, and here's why: It's easier for you and your HR support folks to manage without having to administer multiple performance management processes for the year. That becomes confusing to your pilot stakeholders, participants and your HR support team. It also provides better scope for how your solution serves the various roles and working styles in a full customer group. You're also building collective muscles for the department. If you can include just one department in your first pilot, I would do that. Pick one that represents the variety of roles you want to ensure the CPM design effectively supports.
If there is not one department that provides a willing, nice and tidy group, that's ok. Pick a couple that reach your 10% cross-section and provide enough diversity to gather the data that you need. You will inevitably run across situations that don't fit into a neat, tidy participant group package. All good. Do what you need to move the pilot forward and reduce complexity. There's no right or wrong answer for continuous performance management or pilot groups. And isn't that a beautiful thing? You'll just need to be ready for managing complexity. I lean towards simplicity, as that's what I've found drives the best results and effectiveness, but I also want you to get the data you need. It's a pilot, we are proactively planning for failure, so keep that forward momentum.
Once you've selected a participant group, be sure to get their buy-in. They need to select in, too! Hopefully you've created positive mojo in your alignment work, so you've got several departments lined up who are interested. Be selective - you don't have to include them all. Pick the group(s) that will yield the most constructive data. Plus, you're creating healthy tension and interest in jumping in to the program when you are ready for them.
Now that you've settled on a participant group, they will be eager to get started. Let's ensure they are well prepared for this adventure.
Participant Preparation
Set your customer up for success with a well prepared set of tools and resources. Let's begin with a thoughtful communication plan.
I am a huge fan of articulating communication plans. I find that we tend to have excellent ideas to communicate about this project or that engagement survey. But we don't actually write down the plan. We get busy, we run out of time or are off to the next task. Everything we do - everything - is predicated on our customers knowing what to expect and what to do. Making time to build a communication plan will save you a headache and ensure you are getting what you need out of this pilot.
Communication Plan - People need to know exactly what to expect from the pilot program. Exactly. You want them to clearly know what they will be doing, how the increased discussions they'll have with their leaders will be used, how they will be rated, what form they will use, everything. Ensure you have thought through as many questions as you can from the participants points-of-view. What would you want to know? Answer that. Put it in an FAQ. Doesn’t have to be fancy - just be sure questions are answered clearly.