ORB International Surveys in Syria are Probably Worthless.
[ Posted by Lara Keller 2/10/17 ]
The UK public opinion survey company ORB International (Opinion Research Business) has conducted several polls inside Syria since 2013 (or is it 2014?). ORB International is owned by Gallup and is described as a “respectable” opinion poll company. They specialize in conducting public opinion polls in foreign conflict and other crisis zones. Their polls have been widely reported in the English speaking media as respectable realistic surveys of Syrian public opinion.
Criticism of ORB mainly relates to 2007 when it used surveys to estimate the number of people who died as a result of the Iraq War. There is little criticism of its surveys in Syria.
In 2015 UK BBC commissioned ORB to conduct a survey inside Syria. The part of the poll that covered Islamic State controlled Raqqa, was criticized in a “Radio Free Europe Radio Liberty (RFERL)” article ( https://www.rferl.org/a/raqqa-happiness-poll-syrian-activists-protest-bbc/27256546.html ) : “While most Syrians said IS had a somewhat negative or completely negative influence, a majority — some 70 percent — of the 53 people polled in IS-controlled Raqqa said IS had a somewhat positive or a completely positive influence on events in Syria.”
A Syrian activist Abu Ibrahim Raqqawi described these results as “crap”. ORB would not provide RFERL with information about the polling methodology used in Raqqa (being parsimonious about methodology information was also a complaint in the ORB Iraq deaths survey). Johnny Heald who runs ORB told the BBC that IS agreed to the poll because “as the data verifies, many of those living in Raqqa now are happier since IS took over.” This suggests ORB sought permission from IS to conduct the poll. Since Islamic State is also widely reported as ruthlessly suppressing free speech, it seems highly unlikely the people surveyed were chosen at random and allowed to freely express themselves.
As the Assad regime has also been widely reported for over forty years as ruthlessly suppressing free speech, it seems highly likely the same problem applies to ORB surveys in regime controlled areas. According to the 2015 survey 67% in regime controlled Damascus felt Bashar al-Assad had a completely positive influence, while 23% felt Assad had a “somewhat positive influence”. In Idlib the figures were 5% and 4%.
There is not much on the ORB International’s slick website about methodology. The BBC article on the ORB poll it commissioned contains more information ( http://www.bbc.co.uk/news/world-middle-east-34173549 ): “ More than 14 supervisors and 40 interviewers travelled throughout the country to collect data. ‘It starts with one week’s training in southern Turkey where the supervisors come to Gaziantep and we go through the methodology, the questionnaire and the quality control procedures,’ Mr Heald said. ‘We pilot the questionnaire before it is fielded. We then ensure we have the relative permits/permissions to operate and undertake a risk assessment.’ ”
It seems likely then that ORB conducted their surveys in regime areas with Assad Regime permission, and must assume with the regime’s supervision in some form. Therefore negative or even neutral attitudes to Bashar al-Assad could not have been freely expressed to the ORB survey, and so it is reasonable to question if these parts of the survey have any meaning.
It also reasonable to suggest that different standards apply to the expected reliability of public opinion polls conducted domestically as opposed to those conducted abroad. In the same way as they do to the reporting of domestic and foreign news. A news reporting organization in the UK like the BBC is held to higher standards of impartiality in domestic reporting than in foreign reporting. In practice it is not expected to always be journalistically objective, but only to balance differing popular domestic opinions. There is no large constituency of pro-Syrian Revolution opinion in the UK, which means the BBC can commission and report dubious surveys from ORB, and not get any backlash. Indeed ORB’s reputation has not suffered in the media market due to their probably highly inaccurate Iraqi death toll survey. As Johnny Heald blithely said it was only an estimate. These estimates influence popular support or opposition to foreign policies, and so potentially effects the lives of millions of people.
The effect of the ORB poll has been reflected in countless internet article like this: “Suppose a respectable opinion poll found that Bashar al-Assad has more support than the Western-backed opposition. Would that not be major news?” ( https://news.alayham.com/content/suppose-respectable-opinion-poll-found-bashar-al-assad-has-more-support-western-backed ). The impression of ORB’s Syria surveys is to reinforce the idea that Assad or Islamic Extremists are highly popular with ordinary Syrians, and so the West’s intervention should be avoided. This closes down the popular debate that is urgently needed on good and bad intervention, which should keep the brutal dictators and the Western vested interests happy.