There are quite a lot of misleading panel metrics being used in our industry and people often get confused, when we tell them we don’t pay much attention to them. Sure, we can talk about panel size or attrition rates, but at the end this is nothing we would worry about.

Let us explain what is meant by that. From our point of view there are typically bad panel metrics:

• Panel size: Anyone can pile up a huge amount of email addresses, but that still does not mean you can gather valuable information from their owners. The addresses might not be used anymore, or they are used but survey invitations are directly forwarded to the spam folder. And even if the owner notices your invitation, that does not mean he responds to it. Size means nothing. Huge databases may even be a indicator for bad maintenance, when old and unresponsive accounts were never deleted.
• Panel mortality: The same is true for panel attrition. Only a minority of the panelists unsubscribe from a panel, most of them simply do not respond anymore on invitations. When you exclude these addresses from your panel, you’ll get a high panel mortality and a low panel size, although the quality of the remaining addresses is far better. Without database maintenance you will have low attrition rates and bigger panels. But you will have bad response rates and forecasts for your studies.

So let’s talk about what we consider to be good panel metrics:

• Net reach: Though a calculated value, net reach gives you the maximum amount of interviews you could expect within a specific socio-demographic group. To have 10.000 male registered panelists doesn’t mean anything, but knowing that you could expect a maximum of 5.000 interviews among men is clear and comprehensible. Size multiplied by response rate is the number you should care about.
• Amount of invitations per panelist per month: Big panel suppliers with offices all around the world run far more projects than local companies. Their panels have to lift more interviews and thus have to be bigger. But this doesn’t mean single panelists get invited less often and thus are less used to being surveyed. The number of invitations per panelist  in a certain period of time is the number you should care about. Just register at a panel (if it’s a open registration and count the invitations you receive. You’ll see huge differences!

This are just some examples. We would be happy to get asked more often about the numbers that really matter.

Do you know other misleading metrics? Let us know in the comments!