I wrote Primary versus Secondary Research based on some confusions I was having at a recent conference I attended at which several people presented research. It also caused me to create a new category, Methods, for my posts. This time I'd like to share a bit about a research methodology that is making an erroneous claim -- undocumented success is luck, it's not duplicatable except by more luck and luck is not a good business or research paradigm.
You can never duplicate luck. What you can do is document how luck occurred. This, of course, takes some forethought and planning.
I'm always impressed when I hear or read case studies. They're excellent examples of historical research.Historical research.
As in, we know this happened, we're documenting that it happened. We may be lucky enough to actually interview some of the people who were actually involved in what happened. But nobody actually knew it was going to happen and documented what they did as they did it, there were no plans, per se. We basically felt our way along.
My experience has been (and this is true for NextStage, too) that even when a plan is involved, it's modified as things proceed and nobody documents these many minute course corrections. I think this is why best practices are best practices. Over sufficient time and use the results will fall into some ±2db set. It doesn't matter that these results aren't the best possible, only that it's the most reproducible, meaning we know our luck to within that ±2db margin.
All of which is fine and good, except what do you do with those studies in which the ±2db margin is 75%?
So by all means do share your case studies. Recognize them for what they are, a documentation of something that worked and really, truly, facts-be-told you're not 100% sure about the why or how.
Also share your case studies if you could duplicate a process -- oh -- let's say 5-10 times and get something that approaches a standard deviation in the results.
Just make sure you note your luck factor.
And if you do happen to plan things out and thoroughly document everything that happened along the way and where you deviated and how and why and so on and so forth...ah, well. I'd love to read through your research. That's scientifically duplicatable, defendable, publishable stuff.
Please contact NextStage for information regarding presentations and trainings on this and other topics.
Upcoming Trainings:
- This is Your Brain on the Internet at Emetrics SF on 8 May 2008
- Know How Someone Is Thinking in 10 Seconds or Less Half-day training at the Fashion Institute of Technology in NYC, 13 June 08
- San Francisco Emetrics Marketing Optimzation Summit, 4-7 May 08
- International Communication Association's Communicating for Social Impact, the 58th Annual Conference of the International Communication Association at Le Centre Sheraton Hotel, Montreal, Quebec, Canada, May 22-26, 2008
- SUNY Marketing Professionals Conference at the Fashion Institute of Technology in NYC, 11-13 June 08
Sign up for the NextStage Irregular, our very irregular, definitely frequency-wise and probably topic-wise newsletter.
