A
recent discussion on a friend's wall involved a 2014 survey finding that 92% of
Saudi Arabians believe "Islamic State conforms to the values of Islam and
Islamic law."
Wei
Hong: As you know, study results depend not only on reality but also
on methodology. It is therefore important to examine how questions were phrased
and percentages calculated. Further details of the Al-Hayat survey do not
appear to be readily available, other than that it was conducted via social
media, a method known to introduce large selection bias. For now, the
extraordinary claim of 92% does not appear to have commensurately extraordinary
evidence.
My
friends argued that it is acceptable to use suboptimal evidence as long as one
is aware of the limitations, correctly pointing out that this is often a
practical necessity.
Wei
Hong: While
there does exist a necessary epistemological continuum, it does not actually
follow that poor evidence must be better than no evidence. Medical researchers
recently demonstrated this truth by intentionally using poor methodology to
find that patient outcomes in a cardiology trial (coincidentally named ISIS-2
too) varied significantly according to astrological star sign. Even without
studies showing the contrary, it does not follow that doctors should pay more
than zero heed to astrology. While a poorly designed study can show essentially
any result, we must remember that such result may not only fail to show truth,
but also succeed in showing anti-truth, such that poor evidence can also be
worse than no evidence. (Those who suspect that a large proportion of Saudi Arabians
supported Islamic State in 2014 should also consider that Saudi Arabia has
participated in air strikes against Islamic State since 2014.)
