A pilot program allows you to learn on the fly as you test out the design elements of your continuous performance management program.
Continuous Performance Management Step #4: Learn on the Fly
You are on a journey… a journey to design and implement a continuous performance management process for your organization. You've come a long way. So far, you've:
Designed a basic program, fleshing out your initial ideas and setting guiding principles
Aligned with your stakeholders, developing the start of a business case to use throughout the project
Discovered what your customers need from a performance management process and developed a value proposition statement
You've also crafted your thoughts into a "final, final draft" that you are ready to test. It's time to celebrate your wins thus far. *pause here for the happy dance*
While we pause (and maybe - definitely - dance), let's get oriented. We are on Step 4 - Pilot of an eight-step process to design and implement continuous performance management.
I am passionate about performance management and was so pleased that it was the most requested topic in the 2021 content survey. I believe a solid performance management process that features frequent check-ins between an employee and manager can be a game changer for business results and employee engagement. After all, a 2016 report from CEB indicates that 95% of managers are not happy with their companies’ performance management process and 56% of employees share that they don’t receive feedback on how to improve their performance. Surely we can get a handle on this, providing a better process for managers and employees.
Pilot Testing
Ok now. You've celebrated Design, Alignment and Discovery. Now it's time to launch a pilot test to see how your initial idea fares.
Hopefully the design you've developed is sound and you're making tweaks during the pilot. Congratulations! If you're finding that the feedback is more than tweaks, don't despair. It's perfectly normal and ok! That's why we engage pilot programs. It gives us permission to test out an idea and go back to the drawing board if it's not just right. Don't give up… it's all part of the process.
“I have not failed. I've just found 10,000 ways that won't work.” ― Thomas A. Edison
What is a pilot, exactly?
So glad you asked. According to ProjectManager.com, "a pilot study is a small-scale project that is used to test the feasibility, duration, cost and risks of the full-scale project. Through this study, improvements can be made to better the performance of the larger project."
A pilot enables you to test and tweak, test and tweak, test and tweak, and so forth. Just like a large tech project may run a small pilot to test out the feasibility of new software, so are we running a pilot to test out the theories behind our design and iron out the wrinkles throughout the rollout. We want to find out if the design actually supports employee - manager discussion on performance. Does it help employees correct performance issues before they become trends? Do participants understand and use the tools and training you've developed to support the rollout. What about the actual performance discussion form? Does it effectively capture the data you need? You'll be testing out every component of your plan - every detail. You'll discover updates needed and likely find items you need to add into your plan.
Pilot testing can be fun. It's busy, sure. It's also a time of learning on-the-go and agilely adjusting your plan.
To launch an effective pilot, you'll be working through three actions:
Build your plan
Select & prepare your pilot group
Gather participant feedback
Also worth noting, I've adjusted plans on the go in the middle of a pilot when it was clear the design wasn't meeting the mark, and then re-launched the pilot with new design elements. You'll make that call depending on what you are piloting, such as testing variations to a single component in the design or piloting a first go at the whole enchilada. For enchilada platters, I recommend allowing lenty of time for participants to get used to the plan before changing up elements. Once you've settled on the main design components that are working, you might re-pilot a few elements to see what works best for your customers. The point being, you will want to be agile in how you apply pilot testing. It's an excellent tool to master for your talent development toolbelt.
First things first, let's plan.
Build a Smashing Pilot Plan
Your plan should outline three important details:
1. Duration - How long will you run the pilot?
The most important consideration to project duration for continuous performance management (CPM) pilot programs is to include enough time for participants to learn, use and get comfortable with the new approach. As with any new program, it takes time for your customers to get the hang of things which then allows them to offer balanced feedback.
For CPM pilots I recommend a duration of one year with quarterly milestones to assess effectiveness and acceptance. Because we want performance management to be something employees talk with their leaders about frequently, it will take time to embed the practice into the day-to-day, set new expectations for the timing of discussions and build performance management muscles. I like quick progress too, but give it the time it needs to really make an impact and learn what you need to learn about how it's working.
If you have done an effective job of garnering stakeholder support, once they begin seeing incremental improvements throughout the pilot, you will have other customers clamoring to participate. That's an awesome thing. But hold your ground and stick with your plan. You can include a selection of those interested customers in the next pilot phase.
I have not tried using quick burst pilots to assess the overall program for the above reasons. I have used more frequent changes to weigh the effectiveness of a particular template or training. It's important to know if there are barriers to using a particular template or training, for example. You'll mark those milestone questions down in your timeline, which takes us to the next plan detail to design.
2. Timeline - What are your plan specifics?
You know you love a good plan. If you don't you should! Setting feasible timelines is key. Once you've set the duration, lay it out on paper / whiteboard / somewhere you are comfortable brainstorming dates and pilot steps. Begin plugging in milestone dates such as:
Deadline dates for performance check-ins throughout the year
Annual performance roll-up rating deadline, if you're testing that
Performance calibration timing for merit decisions
Surveys and focus groups (we'll talk more about those below)
Stakeholder updates
Communications you want to share along the way
Program Input Team updates
If you like a Gantt chart, great, lay it out. Prefer PowerPoint? Great, lay it out with a crisp design. Comfortable in Excel? Fab. Record it there. Lay out the timeline, also tracking owners and actions needed to meet the milestone, so that you can track it like mad, staying on course and attending to the details.
3. Participant Group Size - How many will you include in the test?
A pilot is by design a small segment of your overall population. Your program might be picking up speed and getting popular. Don't be tempted! Focus on an appropriate pilot size.
I recommend keeping it small for the first pilot. You need enough folks to gather meaningful feedback, but not so many that you're managing too many expectations and juggling the administration that comes with a pilot program.
What size will get you a solid 10% representation of your total population? Shoot for that. Even better if you can include a whole department that will represent a majority of the roles you need to support with performance management.
Participant Selection is Key
In the last section we began the conversation about the size of your participant group. Let's shift the discussion to how to select a pilot group and ensure they are well prepared.
I mentioned including a whole department in your pilot, and here's why: It's easier for you and your HR support folks to manage without having to administer multiple performance management processes for the year. That becomes confusing to your pilot stakeholders, participants and your HR support team. It also provides better scope for how your solution serves the various roles and working styles in a full customer group. You're also building collective muscles for the department. If you can include just one department in your first pilot, I would do that. Pick one that represents the variety of roles you want to ensure the CPM design effectively supports.
If there is not one department that provides a willing, nice and tidy group, that's ok. Pick a couple that reach your 10% cross-section and provide enough diversity to gather the data that you need. You will inevitably run across situations that don't fit into a neat, tidy participant group package. All good. Do what you need to move the pilot forward and reduce complexity. There's no right or wrong answer for continuous performance management or pilot groups. And isn't that a beautiful thing? You'll just need to be ready for managing complexity. I lean towards simplicity, as that's what I've found drives the best results and effectiveness, but I also want you to get the data you need. It's a pilot, we are proactively planning for failure, so keep that forward momentum.
Once you've selected a participant group, be sure to get their buy-in. They need to select in, too! Hopefully you've created positive mojo in your alignment work, so you've got several departments lined up who are interested. Be selective - you don't have to include them all. Pick the group(s) that will yield the most constructive data. Plus, you're creating healthy tension and interest in jumping in to the program when you are ready for them.
Now that you've settled on a participant group, they will be eager to get started. Let's ensure they are well prepared for this adventure.
Participant Preparation
Set your customer up for success with a well prepared set of tools and resources. Let's begin with a thoughtful communication plan.
I am a huge fan of articulating communication plans. I find that we tend to have excellent ideas to communicate about this project or that engagement survey. But we don't actually write down the plan. We get busy, we run out of time or are off to the next task. Everything we do - everything - is predicated on our customers knowing what to expect and what to do. Making time to build a communication plan will save you a headache and ensure you are getting what you need out of this pilot.
Communication Plan - People need to know exactly what to expect from the pilot program. Exactly. You want them to clearly know what they will be doing, how the increased discussions they'll have with their leaders will be used, how they will be rated, what form they will use, everything. Ensure you have thought through as many questions as you can from the participants points-of-view. What would you want to know? Answer that. Put it in an FAQ. Doesn’t have to be fancy - just be sure questions are answered clearly.
We'll talk more in Step 5 - Over Communicate about building a change communication plan for the implementation in full. For the pilot program, leverage your roadshow collateral and the timeline to be very specific with the groups on what they can expect. While I advise that pilot communications be sharp, I would rather you produce the communications than worry about the perfection of them. Let's go with the 80/20 rule here.
The first group to communicate with are the leaders of the workgroup. You are likely changing up how they manage their teams. Build buy-in with them first. Then you're ready to cascade information down to everyone in the group.
Here's your checklist to consider for your communication plan:
Note from the department / group leader
Note from you or their HR advisor with an introduction and expectations
Roadshow overview to give them the high-level; this is a great virtual + optional offering
Timeline and key dates
Frequently asked questions (FAQs)
Reminder notes throughout the pilot
Training Materials - Another important prep element is training on how to engage the new process. I've found that even when you design for user-friendliness, we still need to provide thorough training and support materials. Performance management is not intuitive to most, probably because it can be challenging to give and receive. So take any guess work out of the expectations. You might not need a comprehensive training program, but what about a job aid for employees and one for leaders? A sample of what a completed check-in from should include is helpful. Run ideas and samples through your Program Input Team.
Here's a list of training collateral I've seen that work well:
How-to job aid for employees
How-to job aid for managers
How to complete the form in the system / How to upload the form in the system
Short video intro with a few words on why and the WIIFM (what's in it for me?)
Super quick elearning module
Completed sample template
Performance conversation starters
Mid-year meeting with managers on how to conduct performance conversations
You'll find that you receive a lot of feedback about the training materials you provide. Don't be discouraged. That's exactly the kind of things you are looking to learn about, especially should you find that you aren't getting a lot of input on the overall design and it's working well. Feedback on the training or communication materials will enhance future pilots or implementation.
And We're Back to Gathering Input
You are old hat at gathering customer input by now. Remember when I mentioned in Step 3 - Discovery that you would be using the surveys and focus group skills you built to gather customer input during the pilot phase? Whelp, here we are.
Online Survey
I recommend building into your timeline an online survey after each appraisal or check-in step of your process. That allows you to see progress over time, as well as feedback on any updates you made to the approach for each cycle. You should also plan for an online survey at the end of the program.
Consider these sample statements (rated 1 to 5) for a pilot of a quarterly CPM process:
The new quarterly form helped me have a better development discussion with my leader.
After a check-in with my leader, I had a clear understanding of my performance and what I needed to focus on for the next quarter.
Setting priorities on a quarterly basis made them more obtainable than setting annual goals.
I understand why we are making the move from an annual performance appraisal to a quarterly check-in.
Focus Groups
Focus groups I would reserve for one, maybe two, times within the pilot timeline. It can be costly and difficult to get employees together for focus groups, so use that tactic wisely. What you're looking for with focus group are qualitative inputs that verify or give more depth to the trends you're seeing in a survey. You might also use them to work through specific elements of the design, like the check-in form.
You may factor in other ways to collect feedback. Catch folks in the hallway for anecdotal feedback. Check in with leaders at a staff meeting. All feedback is valuable and should be considered a data point.
Hopefully you've worked into your timeline when you plan to circle back with your Program Input Team or others who can help you incorporate updates into the program design, because it's about that time…
That's a Pilot Program Wrap
Success! It's time to celebrate another win! Completing a pilot program is no small feat. You've come a long way in the design and implementation of a process that will support your organization, managers and employees.
Now it's time to pause to build in the feedback to your program design. Pull together a summary of your findings with the pilot program. Decide what elements of the design work well and which need updates. Meet with the Program Input Team to share and discuss the results. Further, what are your next best steps? Would a second pilot program be valuable to engage another layer of feedback. What's the appetite in your organization? More testing to be sure or do they lean towards "let's give it a go?"
Plus, you'll need to update the program collateral, such as your roadshow deck and share the good news with your stakeholders. We'll talk more about that in the next Step 5 - Over Communicate.
You've accomplished a lot in the Pilot step, and likely it's been a chunk of time in your overall implementation plan.
Quick Summary for Pilot
During this step:
You've developed a plan for a successful pilot program, including a detailed timeline.
You've identified a group to participate in the pilot and prepared them well with more documentation than they think they need.
You've collected data - both quantitative and qualitative - on what worked well and what needed an update from the pilot participants
Hopefully you've been continuing to keep your stakeholders abreast of the pilot results as well as continuing to educate the organization as to the awesomeness of your plan. We've got you covered in the next step - Over Communicate - on how to do that well.