How to Build Your Own Norm Group for Hiring
Generic norm groups say little about your specific context. Learn how to build your own norm group that makes your hiring data truly valuable.
Door Ingmar van Maurik · Founder & CEO, Making Moves
Why generic norm groups fall short
When you use assessments in your recruitment process, the results are always compared to a norm group: a reference population that determines what constitutes a high, average, or low score. Most assessment vendors offer generic norm groups based on thousands of people who have taken the test.
That sounds solid, but there is a fundamental problem. That generic norm group contains people from all sorts of industries, functions, levels, and countries. A score of 70 on a cognitive test might be excellent for one role but subpar for another. It is like comparing a marathon runner's performance with the average of all athletes, including swimmers, weightlifters, and chess players.
For companies serious about data-driven hiring, building your own norm group is not a luxury but a necessity. In this article we explain how to build one, step by step.
What exactly is a norm group
A norm group is a collection of scores from a specific population that serves as a reference point for interpreting new scores. When a candidate takes an assessment and receives a score, that score alone says little. Only when you compare that score to a relevant norm group do you know whether it is good or bad.
The elements of a norm group
|---------|-------------|---------|
Generic vs. specific
A generic norm group is broad and readily available. A specific norm group is narrow and must be built yourself. The difference in value is enormous:
Generic norm group: tells you a candidate scores better than 70 percent of all people who have ever taken this test. Useful as a baseline, but not informative enough for good decisions.
Own norm group: tells you a candidate scores better than 70 percent of successful senior developers at comparable companies. Now you actually know something.
How to build your own norm group
Step 1: define your target population
Start by clearly defining who you are building the norm group for. Be as specific as useful:
The more specific, the more valuable the norm group, but also the longer it takes to collect sufficient data. Start broad and refine as you gather more data.
Step 2: collect assessment data
The core of your norm group is assessment data from candidates who have gone through your process. Ideally you collect:
It is essential that you collect data from all candidates, not just hired ones. If your norm group consists only of hires, it is skewed and not representative of the actual distribution.
Step 3: determine the minimum size
A norm group must be large enough to be statistically reliable. The rules of thumb:
At 50 hires per year for a specific function category, you have a good norm group after 2 to 4 years. Too long? Then start broader, for example all technical functions, and refine later.
Step 4: calculate norm statistics
With sufficient data, you calculate the statistics that define your norm group:
Mean and standard deviation form the foundation. They tell you what the typical score is and how large the spread is.
Percentile scores translate raw scores into a position relative to the group. A percentile score of 75 means the candidate scores better than 75 percent of the norm group.
Score bands define what is low, average, and high for your context:
|----------|-----------------|----------------|
Step 5: validate against performance outcomes
A norm group becomes truly valuable when you validate it against actual performance. This means investigating the relationship between assessment scores and:
If you discover that a score of 65 on a specific assessment strongly correlates with success in the role, you know that 65 is a meaningful threshold for your norm group. This is the foundation of predictive hiring.
Step 6: maintain and recalibrate
A norm group is not a static product. You need to regularly update it:
The role of technology
Building and maintaining your own norm group manually is an enormous task. You need data from multiple sources, statistically correct calculations, and continuous monitoring. This is where a custom hiring system proves its value.
A good system:
Without such a system, you depend on spreadsheets, manual calculations, and the goodwill of someone who understands statistics. That is not scalable and not reliable.
Common mistakes
Only including hires in the norm group: this leads to range restriction. You only see the top of the distribution and miss the full picture.
Using too small a norm group: with fewer than 50 scores, your conclusions are statistically unreliable. Wait until you have enough data or start broader.
Never recalibrating: your norm group becomes outdated as the labor market changes, your company grows, or the role evolves. Plan at least an annual review.
Using one norm group for everything: different roles require different norm groups. A norm group for developers is not usable for product managers.
Key takeaways
Want to start building your own norm group? Get in touch or read how generic assessments fall short and why customization is the future.