Can you trust the results of TNNA’s State of Specialty NeedleArts surveys of needle arts enthusiasts and make business decisions based upon them? Yes. The survey questions are valid and reliable, the number of responses statistically significant, and the sample valid.
All of the questions on The State of Specialty Needle Arts surveys (and any other Hart surveys) have been pretested to ensure they are valid and reliable. This includes administering the questions to people one-on-one and making sure each question is easily understood; if a question is confusing we rewrite it until it is understandable.
The spending questions are intentionally top-level. People would have difficulty remembering exactly what they spent on knitting needles, for example, so spending questions are kept more general. People also get frustrated if questions are too detailed. Ideally the study would pay people to keep spending diaries or conduct monthly spending surveys, but then research costs would rocket to $250,000 or more, which would be crazy for the needlearts industry. (The cost of the TNNA study was way, way less than that.) The per needle artist spending medians and percentages per product category have been validated by the industry and by major retailers, who say it is close to what they see in their own businesses. The spending percentages per product category and spending per person have been consistent over the nine years we have been conducting the surveys.
12,000 responses to a survey (in the case of knitters) is way more than needed, but is great for looking at slices of the market. 450 responses is enough for a valid sample, assuming the sample is not biased. Given the range of sources that contributed respondents, the knitter and crocheter response sets are a valid sample of knitting and crochet enthusiasts. Remember these are enthusiasts and not just any person who picked up a knitting needle in the past year. Also, note this was an online survey available to anyone, so we did not control where responses came from. We did not track IP addresses or referring URLs, to protect anonymity. People reposted the survey links all over the place. The survey did ask people what Web sites, Facebook pages, etc. respondents used, and they were a wide range.
So, in sum, The TNNA State of Specialty NeedleArts studies use a cost-effective research methodology that generates data you can depend on for good business decisions. For more information see the study overview.