See, The Thing About Statistics is…

Open Letter to Utah State Representative Carole Moss (D)

Dear Representative Moss,

On Friday, January 27th, during the House Rules Committee there was discussion regarding HCR 11 – a resolution urging the President to rescind the Bears Ears National Monument. During that meeting, you mentioned two statistics: 1) 66% of Utah is in favor a new national monument for Bears Ears, and 2) 64% of all San Juan County residents commented in favor of Bears Ears. I would like to address each statistic, as I feel it’s extremely important that you learn the background and methodology behind each one.

The first was a Western States Survey conducted by Public Opinion Strategies. This particular poll surveyed only 400 registered voters in each state, and asked over 30 vaguely worded or baited questions with sliding scale response options (i.e. Extremely serious, Very serious, Somewhat serious, Not serious, Don’t know). The following is the exact question taken from the survey that produced the 66% response in favor of the monument:

Question:
“Some of the public lands in Utah are designated as national monuments, in which lands are protected from development, mining and other kinds of resource extraction. One proposal which originated with local Indian tribes is to protect nearly two million acres of existing public lands surrounding the Bears Ears Buttes south of Canyonlands National Park as a National Monument, in large part to protect cliff dwellings and sacred American Indian sites. Would you support or oppose designating these existing public lands as a national monument?”

Responses:
39% Strongly Support
26% Somewhat Support
8% Somewhat Oppose
12% Strongly Oppose
14% Don’t Know

The way this question is worded, one might think that if they admitted to opposing a monument, they would be marked down as not wanting to protect public lands or support Indian tribes in protecting their sacred sites. I’m not a lawyer, but isn’t that what they call “leading the witness”?

The second statistic, which both the Utah Dine Bikeyah (UDB) and Bears Ears Inter-tribal Coalition (BEIT) promote on their websites (http://utahdinebikeyah.org/letters-to-the-editor/ and http://bearsearscoalition.org/timeline/, respectively), claims that 64% of San Juan County residents support the Bears Ears proposal. Because neither website provided any links to verify this data, I researched it myself. I was able to find minutes and a recording from a San Juan County Commission meeting held on December 8, 2014 that discussed the results of a “San Juan County Lands Bill Open House Events and Public Comment” survey. It was reported that a total of 468 comments were collected from the open house events held in October and November of 2014. Of the 468 comments, 300 of those supported the monument (there’s your 64%!). However, it needs to be noted that of those 300, only 194 were actual SJC residents. The remaining 106 comments came from surveys with out-of-county/state mailing addresses or could not be verified any other way (i.e. not signed). Since the other proposal option results did not break down the comments by location, there is no way to verify just how many of the remaining 168 comments were from actual SJC residents. Furthermore, the summary reported that it ‘may include some duplicate comments from the same commenter’. To be accurate and honest, the UDB and BEIT should have stated the truth on their websites, which would be “64% of the 468 individuals who responded to the survey made comments supporting the monument.” But claiming a general “64% of San Juan County residents support a Bears Ears National Monument” is not only false, it’s misleading and it’s a calculated attempt at causing division within the county, state, and between Native American tribes. If the opposition to the monument (which I am a part of) wanted to play that game, we could easily refute that statistic and argue that “only 194 out of 14,973 SJC residents have expressed written support of the monument.” Wow, now wouldn’t THAT be an amazing headline?

‘BREAKING: LESS THAN 2% OF SAN JUAN COUNTY RESIDENTS SUPPORT BEARS EARS NATIONAL MONUMENT!’

It’s catchy, but somehow, I don’t think the Salt Lake Tribune would pick up that story.

Many of us in SJC have wondered how and where these polls (ones that have reported 50% or more in favor of the monument) were coming from, because we simply know that the majority is opposed. They are classic examples of what you might find in a “Survey Manipulation for Dummies” textbook. Because of that, the Stewards of San Juan County decided to conduct a survey of their own back in September 2016. This survey was posted online, shared via social media and email, and hand-delivered throughout the county. We made it a point to share with as many people as possible, even with those that we already knew supported the monument. You can find the article here. In one week we received 1197 responses, and almost half (572) were from SJC residents. Of the 1197 surveyed, only 49 individuals (4%) either supported the monument or were undecided. Of the 572 SJC residents, you might be surprised to learn that all but eight people opposed the monument…an overwhelming 98%. Personally, I would not feel right about making either one of those specific claims, on either side. I would, however, be comfortable saying that between those two surveys, 564 San Juan County residents who have expressed written opposition to this monument is a greater number than 194 who have expressed written support for it. Wouldn’t you agree?

Do not get me wrong, I can’t say that any of these surveys have been conducted in a real scientific and professional manner. True statistics need to be done in a way that includes proper collection, analysis, interpretation, presentation, and organization of data, and they have to be done without bias and emotion. But that’s the hard part, you see, because Bears Ears is an emotional issue, on both sides of the coin, and even you stated that in your testimony on January 27th.

I just have one request. Before your next house committee hearing, and before you decide to rattle off a couple of statistics that you found on a website (or maybe it was Senator Dabakis that was kind enough to forward them to you), please do some serious research behind that data. They might be just silly little numbers that are thrown around in the media and used for personal gain, but unless fully and completely investigated, those silly little numbers – promoted by the wrong people, the wrong organizations, and in the wrong context – have the potential to destroy many families, communities, and lives.

I appreciate your time and consideration on this matter.
Sincerely,
Jami Bayles, President
Stewards of San Juan County

2 thoughts on “See, The Thing About Statistics is…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s