The American Horse Council is a longtime strong advocate for equine endeavors in the United States. The organization’s efforts in the nation’s capital have been invaluable…so I write this post in the spirit of helpfulness rather than criticism.
Currently, the AHC is conducting a “horse owner and supplier survey to update the [2005] National Economic Impact Study.” Unfortunately, the study will not achieve the desired result because its data-collection methodology is manifestly unscientific.
On July 19, 2017, the AHC put this notification on its website:
“The American Horse Council Foundation (AHCF) announced today that it will be extending the deadline to complete the horse owner and supplier survey to update the National Economic Impact Study [of 2005] to August 18th.
‘The survey has been open since the beginning of June, and unfortunately we have only had around 9,000 responses,’ said AHC President Julie Broadway. ‘We decided to extend the deadline for respondents to take the survey to ensure that we are getting as many responses as we can—we estimated that the survey link should be reaching approximately 900,000 people, and the 2005 Study had over 18,000 responses itself. The industry has waited a long time for this study to be updated and we want to be sure we are getting the full picture of the impact of the vast equine industry.’”
The AHC survey is fatally flawed in that it is online and anyone can fill it out. Therefore, “getting the full picture of the impact of the vast equine industry” is not possible.
The sample is what is called a “convenience sample,” which is typically used for exploratory research and hypotheses formulation rather than to make statistical inferences. The self-selection and Internet-collection procedure ensure that respondents in total are unrepresentative of the horse-owner (and supplier) population in the United States. Not all horse owners are computer literate and some unknown percentage who are computer literate will not respond to the survey; it is likely that the majority of horse owners and suppliers to the industry may not even be aware the survey is underway.
In any survey research, far more important than how many people respond is whether the sample is representative so that it can be projected to the larger population from which it is drawn. Far better to have a random sample of 1,500 U. S. horse owners (and suppliers) than thousands of self-selected respondents to an online survey to achieve what goes by the technical name of “external validity,” or the ability to make generalizations about the larger population.
To demonstrate how mistaken survey results can be if a study is not properly conducted, the AHC research consultants in the 2005 economic-impact study extrapolated from their survey sample that there were between 8.9 million and 9.6 million horses in the U. S., with a midpoint projection of 9.25 million. Of these, 1.3 million were said to be Thoroughbreds. [The study was commissioned in 2003 and published in 2005].
Simple calculations prove conclusively that the AHC projection for the number of Thoroughbreds was an actuarial impossibility. The Jockey Club’s records show that every single Thoroughbred foal born in North America (United States, Canada, and Puerto Rico) from 1969 through 2003 would have had to been alive in 2003 to add up to a population of 1.3 million. (Annual American imports of Thoroughbred horses were not a factor because historically they represent a relatively small number and are exceeded by exports by a multiple of two or three to one.)
Because of the way the data in the 2017 AHC study were collected, one cannot have any degree of confidence that the findings will even be in the “ballpark.” Moreover, there is just as good a chance that the economic impact of the U. S. horse industry will be underestimated as overestimated.
Here’s hoping that the AHC will abandon its present methodology and try again with a methodology that will enable it to make defensible and believable projections about the economic impact of the horse industry in the United States.
Copyright © 2017 Horse Racing Business
Somebody sold the Council a false bill of goods on this study. The researchers should be fired. A state or federal representative reading the results of this study won’t be impressed.