My week: Power to the People

Something important happened this week; something, that empowers the people. And no, I’m not writing about the new pope, I’m writing about the European Parliament, who has rejected the draft budget of the EU. It was probably the first time that it was not a single country blocking the negotiations of all other countries, either with a referendum or a stubborn government. To me it always seemed to be rather unfair that a minority is able to govern the rest of Europe, while the European Parliament, being the only elected institution of the EU, had almost no power. With the “No” to the budget, the European people may be confident of getting an important voice in the future.

Currently we’re working on the website of our panel, trying to enhance the functionality, making it easier for our members to engage with research projects and therefore more likely to participate in surveys. We had a lot of extensive meetings during the last weeks, collected a lot of brilliant ideas in exhausting brainstorming-sessions. It will be great advance, we’re sure. We’re so convinced of our ideas that we don’t need to ask the people. We know what works!

screenshot_op

Well, admittedly, we don’t! Therefore we conducted a study within our panel to find out, how we can improve. We had some good ideas ourselves, but not all of them could convince our panelists and therefore had to be dropped or modified. Some ideas were good, but our panelists helped us to make them even better. And some ideas came from the respondents themselves, knowing the panel from a different perspective than we (and it is probably from the only perspective that really counts).

It is important to mention, that we didn’t wait until the launch of our new website. Our study was not meant to be the evaluation of a definite solution; the contrary is true. It was a first input for our team, before starting to develop something and spending hours on coding worthless gadgets. And we are planning to keep in touch with our panelists during the development process. It is easier to modify a concept when the paint is still wet, rather than when the whole piece is concluded and has become some kind of awkward reality. And I think that’s exactly what Europe is learning at the moment.

We’ll see what next week brings. Happy weekend!

Posted in General, Methods | Tagged , , , | Leave a comment

The Monetary Value of Good Questionnaire Design

Incentives make people doing things they would not do without the incentive. Participate in surveys for example. Or even worse, participate again and again in surveys, unless they’ve learnt that it’s quite boring. But then, could we offer less incentive, if surveys were a more enjoyable experience?

This is exactly the question I tried to answer last week at the GOR conference in Mannheim. And given that we need less incentive, what is the value of good questionnaire design?

The answer to the first question is simple: good questionnaire design may reduce the need for a monetary incentive dramatically. In our study, we were able to save 40 cents per interview while maintaining the satisfaction of our respondents on a high level. So it’s definitely worth to spend some time on questionnaire design.

The bad news is that there is no general formula for fun. Even when people are asked to do the most boring tasks, the can derive a sense and pleasure from it. It’s just a matter of how they frame the task mentally. That doesn’t mean that we are allowed to do dull surveys; we are probably still responsible for enabling a positive experience. But I think it challenges the idea of inherent “good” or “bad” elements of survey design. As is often the case, the dose alone makes the poison.

Posted in General | Tagged , , , | Leave a comment

My week: Sampling and Questionnaire Design

This week I was attending to the 15th General Online Research Conference in Mannheim / Germany and this event always seemed to me like a big mosaic. If you’re too close, you will only see a lot of small brilliant pieces, but if you want to see the big picture, you will have to take a step behind and this may require some time. So I’m not able to give you a summary yet, but it was definitely worth being there.

Instead, I’d like to write about the inspiring workshop of Mick Couper, unless it may be only one brilliant piece of research among many others. He gave a lot of useful background information about the thing we use to call surveytainment. My definition of surveytainment is the uncompromising application of the best practice principles of web survey design – and this is exactly what Mick Couper talked about. So I suppose we’re on the same side!

Nonetheless, I somehow realized that something is slightly different between our points of view and it took me some time to figure out, what it was. I seemed that Mick Couper is far more skeptical about using interactive and dynamic survey elements than me and my impression is, that this is a result of working with different sample sources. I mean, if you work with fresh sample it is very different how you have to treat your respondents than when you are working repeatedly with panelists (and you aim to work with them again and again in the future).

In the latter case, you have to be much more careful about the appearance of the survey, since you don’t want to harm your relationship with the respondent. On the other hand you can rely on this relationship when demanding for an updated browser version or the availability of JavaScript. So maybe the most sophisticated techniques of surveytainment only make sense when working with a panel – but from my point of view there is little reason to worry about it.

Of course, I understand academic researchers being skeptical about surveytainment. There is still a lot of research to be done! Nonetheless, that doesn’t mean that from a practitioner’s perspective surveytainment doesn’t work; otherwise none of the panels would apply these techniques to their biggest asset, the respondents. Maybe there is a lack of scientific documentation, maybe a lot of the relevant factors are unknown and it is absolutely unclear how they work together – but it works. Little empirical evidence (in the strict scientific meaning) does not mean that there is no effect. It only means that there is little documentation of the effect!

We’ll see what next week brings. Happy weekend!

Posted in General, Methods | Tagged , , , | Leave a comment

My week: Predicting the Future

Italy has voted and some Oscars have found a new home. Are you still assessing your predictions of these events, or are you already back to reading the crystal ball again? Benedict XVI resigned! Who will be the next pope?

Among all the fascinating things a market researcher can do, predicting the future has always been the most exciting one to me. Probably this is due to the fact that it is absolutely not possible to predict the future and that every prognosis is necessarily vague. I mean, social research is far from being an exact science, but futurologists do not even pretend that there is a hypothetical possibility of making accurate measurements. The only thing that matters is whether the prediction was right or wrong and as long as you’re right nobody cares about the reasons.

723987842_c8599d2e98_bPhoto Credit: bjornmeansbear

That doesn’t mean that there is no science behind it. I’ve great respect for accurate predictions and I know that electoral forecasts require much more than a certain knack. But I also know that prediction markets are capable of making good forecasts as well; even though this time you need to know completely different things than a pollster. And this is why I’m very open-minded towards social media predictions: I don’t understand the science behind it and probably it would take me ages to do so. But I see that social media researchers are able to make good predictions and therefore I don’t care about the reasons.

That gets me back to the pope. According to an ancient prophecy the next pope will be the last one ever, ushering in the Apocalypse. Bright prospects! Admittedly I don’t understand the science behind this prophecy, but anyway, I don’t believe in it. I don’t have a better or more accurate alternative to offer, but for me the Apocalypse is still hard to imagine (I know that disbelief is a sign of the coming end, but I can’t help!). Hence, plausibility is probably the most important ingredient for a good prediction. Who will be the next pope?

We’ll see what the future next week brings. Happy weekend!

Posted in General | Tagged , , , | Leave a comment

My week: The Horsemeat of Market Research

Whole Europe is shocked about the horsemeat scandal, unless everybody somehow knew about the quality you can arguably expect from low-budget convenience food. And from my point of view this shock has nothing to do with general disgust for horsemeat. (Some horsemeat markets even reported a higher demand during the last days). The shock rather reflects mistrust in the transparency of the food industry as a whole, since wrong labeling pops up in different countries and supermarket chains. Everything seems to be interrelated. And among all the different brands, with all their different promises of unique standards of quality, there seem to be no differences at all!

So what is the horsemeat of market research? Are there any issues that could pop up in different countries and companies? Hum, maybe with survey sample brokerage! I mean, sample brokerage is not disgusting per se – most often it’s not disgusting at all. It may turn out to be exactly what you were looking for. But how will you know? Your data file won’t tell you about the sources of sampling. It won’t tell you about the quality standards and measures of panel maintenance. It won’t tell you about possible biases. The only thing you will see in your file is a decent number of cases. Everything else is a matter of trust, but you will never be able to have a look behind the scenes.

What can you do? You could give preference to a fieldwork provider with proprietary panels, even when looking for sample in other countries. The reason is, that panel provider will be more likely to work with partners that have similar quality standards. For example Norstat, with its CATI-recruited panels, has been our partner in the Nordics until we joined forces, because we wanted to guarantee full comparability between the different data sources. The question remains, why you should work with panel providers, rather than a broker? Because they will be able to give you information at first-hand about their quality policy. They will be able to assess the quality of different providers and benchmark them against their own standards. Just some thoughts on it, what do you think?

We’ll see what next week brings. Happy weekend!

Posted in General | Tagged , , , , | Leave a comment

My Week: Two Opposing Positions

The party is over! Carnival has finished, back to work. Recover from your headache, cure your hangover. It’s time to get serious again!

Really? I’ve met a couple of clients during the last weeks, in the UK and Germany. And it seems that there are two kinds of online researchers. There are those, who take online research quite serious, and those, who don’t take it serious at all. The latter mainly complain about the uneven coverage of internet users among the population and the lack of random samples; according to their opinion online research can never claim to be representative for the general population. The only thing you can arguably do with online samples is to conduct experiments. On the other hand there are a lot of researchers who are very serious about online research. They experienced relevant and valuable insights for improving their (clients) business and they developed several techniques to get most out of their research. They will tell you that online research has a high practical value, and that other methods of data collection have their deficits as well. Obviously, these are two opposing positions.

But to my surprise those who take online research seriously seem to be much more demanding than those who don’t expect any benefits from doing online research. They really take care about quotas, screening criteria, field times, questionnaire design and things like that. They believe that these aspects ensure the quality of their research projects. They want to know everything about your panel, the quality guidelines and they assess the information thoroughly. In contrast researchers of the other kind tell me that quotas do not make a difference. That the effect of poor questionnaire layout is too low to compensate for non-random sampling. That online research is just a matter of cost-effectiveness and that therefore they don’t have any specific requirements for the fieldwork. “Online research can’t provide us with valid results, so why would we make any efforts to brush up something that remains incorrect anyhow?”

Who is right? We’ve worked with both kind of researchers, and they were equally satisfied (of course, they had different expectations). But wouldn’t it be better, if the critics took online research more serious? Wouldn’t they obtain better results? On the other hand, can you overdo online research? And wouldn’t it be beneficiary for the projects to have a rather flexible structure in quotas, field times and so forth? Should the enthusiasts become less rigid about online research?

Of course, I’m exaggerating. Most of our clients are somewhere between these extreme positions. But my question is, how serious do we have to be about online research? And how serious do you take online research? I’m looking forward to your comments…

We’ll see what next week brings. Happy weekend!

Posted in General | Tagged , , , | Leave a comment

ODC Client Day London – The First Meeting of the ‘Secret Society’

IMAG0434

Thursday the 7th of February 2013 was the date of the inaugural meeting of ‘The Secret Society of Market Research’. In light of the many changes happening in and around our industry the ODC UK team decided to bring together some key industry figures and research experts to put an end to some of the myths around the ‘Big Data’ issue as well as offering some interesting tips on how to maintain a competitive advantage in the fast-moving online research market.

The event was attended by a professionals from all tiers of the industry including, EMI Music, BAMM London, Bauer Media, Bryter Research, RS Consulting, AIMIA, SPA Future Thinking and Aurora.  The event was held at a top-secret location (in the private dining room at the Hawksmoor Restaurant, Guildhall) early in the morning with presentations taking place whilst the guests sampled the wonderful variety of breakfasts dishes on offer.

images

Mike Merritt-Holmes and Pinal Gandhi from the Big Data Partnership introduced the Big Data issue to remove some of the hype surrounding the term. We discussed some of the key external, business and technical principles and also some of the ways that business in various sectors have used Big Data. Finally we talked about ways in which it can be used in the market research sector which stimulated some healthy debate among the guests.

Florian Tress, Research and Development Director at ODC presented the ‘Secret Knowledge of Good Online Research’. His talk covered the change and development of the online space including the functionality and complexity of the websites and online content that we use today compared to the late 1990′s. How this has effected the research market looking at the early days of online and its growth through to the present day including the challenges that we face looking forward. The key learning is the need to find a balance between the current ability for respondents to be contacted and complete studies everywhere and at any time but ensure that the quality of the survey experience does not suffer when distributing it across multiple platforms.

Demonstrating ODC’s range of bespoke solutions to this issue Florian featured some new question types that are clear to understand, well designed and very easy to use but also suitable for issue across a wide range of market verticals. Specifically this included great flexibilty with branded studies, raised engagement for low involvement topics and promoted improved data quality.

If you would be keen to learn more, join us at our next event or request copies of the presentations from the day then please feel free to contact me via my email – m.reynolds@odc-services.com

Thanks to all our attendees and I very much look to the next gathering of ‘The Secret Society of Market Research’

Matt

 

 

 

Posted in Events, General | Leave a comment

HAPPY NEW YEAR!

Dear Clients, Partners, Friends and Shareholders,

Shame on me!

This blog entry was supposed to include season greetings and should have been published on December 24th 2012. I have to admit, I simply forgot about it as I left for my holidays a few days earlier. Feeling a bit guilty, I found this one on the internet – enjoy!

http://mimiandeunice.com/wp-content/uploads/2011/04/ME_345_15MinutesOfShame.png
http://mimiandeunice.com

So I hope that all of you had a merry Christmas and some relaxed and enjoyable holidays.

On behalf of the ODC Services Team I wish you just the very best for 2013. We look forward to working with you again in 2013 and wish you peace, happiness, and abundant good health in the new year!

Yours sincerely,
Christoph Irmer

 

 

Posted in Advent Calendar, General | Leave a comment

Christmas songs

Music is the soundtrack of every moment of our lives.
Listening to music at Christmas can immediately put you in a good mood and let you feel the magic and spirit of this time.
While looking for some Christmas songs, I came across this video…And found it very funny! Just play and enjoy.
Merry Christmas!

Posted in Advent Calendar, General | Leave a comment

How I spent the End of the World…

[By the way, there is a Romanian movie called in a similar way but the story refers to a different “End of the World”, namely, the revolution from 1989 when the Ceausescu’s regime finally fell. It was an end of a world for all of us]

Well, since the famous Mayan “End of the World” did not bother to show up this time either – this Diva attitude is so annoying! I don’t even

want to bring up the year 2000 when it stood people up big time!  We all remember THAT day! – leaving people alone in the cold and forcing them to open the dams for the warm endless rivers of gluhwein, we all wonder how the new Mayan Calendar will look like. What will happen next? An era just ended and a new one began? Maybe. Or maybe on the contrary, we will start all over again! Huh! Figure that! In any case, a new calendar just became a mandatory issue.

Well, after a lot of time invested by some guys a

nd gluhwein – this was my part and I’m telling you it wasn’t easy! – I am proud to announce that the new Mayan Calendar for 2013 looks like this (I have to warn the weak hearten that the Calendar is quite bloody….ordinary!):

calendar

 

So don’t forget to mark down all the important dates, just like you did in the old calendars. The system is pretty much similar and user friendly :)

 

Posted in Advent Calendar, General | Tagged , | Leave a comment