A rejoinder to my post on the AL polls

I received a 'correction', from Radwan Siddique (who has been involved in the Awami League election campaign) concerning my blog post, What to make of the Al's poll' which was published a month back

For those who did not follow this story at the time, in a nutshell the post concerned a poll 'commissioned' by the AL, undertaken by a company, one of whose owners is an Awami League MP, with the assistance of an international NGO  There was a difference of opinion between the AL and independent statisticians about how the raw data should be weighted. Two versions of the 'poll results' were in circulation - one of which was published in the Independent and Ittefaq newspapers which showed that AL was in the lead by about 6 percent (the 'AL version'), and another set of results published in New Age which showed that BNP was in the lead by 3 percent (the 'independent statistician's' version).


Radwan e-mailed me the following:

"In your post titled 'What to make of the AL's poll?', there are two factual errors. Firstly, AL did not commission the poll and secondly, the raw data was not analyzed ( I presume you mean processed) by the party. 

The party of course went over the data it was presented and even pointed out a few calculation errors. There was only one layer of weighting which the party and the organisation responsible for the poll agreed to disagree on (hence the discrepancy between the two sets of final numbers). The unweighted figure had AL at 49% and BNP at 32%.
I am sure you will agree that the main thrust of the post relies heavily on the two claims in question and thus a correction is required to avoid readers being misled."

It appears that he was particularly focusing on this paragraph in the blog:

What both of these articles did not say was the the poll was (a) commissioned by the Awami League; (b) the raw data was collected by a company owned in part by an Awami League MP (*see ownership details of company at end of article); and (c) the raw data collected was analysed by the AL's research wing - which is run by Sajeed Wazed, the prime minister's son.

Following his request for a correction, I e-mailed him the following queries

Before I make any changes, can I just clarify a few things:
1. Was the poll done 'for the AL'
2. Did AL chose the polling company
3. Did the AL (or someone on its behalf) pay for the poll to be done? (i.e paid the polling company)
4. Did AL do weighting of the raw data?

He responded to these questions with this e-mail:

1. The poll was done as part of an ongoing [named INGO] programme.
2. The company was chosen in partnership with [named INGO] after taking a number of factors into consideration.
3. AL had nothing to do with the financing of the poll.
4. AL did not do the weighting.
It's not a huge surprise that voters see AL as a safer pair of hands than the BNP in running the country. Other polls also indicate that 2009-14 is viewed much more postively than 2001-06 by the majority of voters. No one is 'cooking the books' here!

I then e-mailed him again:

Just one further query. My understanding, which seemed to be reflected in your first e-mail, was that there was a disagreement about weighting of the data - hence the difference in the analysis which New Age published (that showed BNP up by 3 points) and the one that AL published (which showed that AL was 6 points in the lead) - relating to the same raw data of course. So who did the weighting that was behind the results published in the Independent etc? surely that must have been AL or on behalf of AL - or are you saying it was the polling company?

He then responded in this manner:

The way polling works is that different layers of weighting are applied to the raw data (demographics being the most common). This was all done by [INGO], AL merely checked the numbers, calculations and formulas used. The last layer applied by [INGO] was to adjust the polls for the respondents' preferences in the last elections (if I remember correctly) which is not a common practice especially in South Asia where respondents tend not to admit that they voted for the losing side. The Independent poll didn't include this layer.  

I am always happy to make corrections when errors have been identified, so lets look at these claims, and see whether anything needs to be corrected.

Q1 - did the AL commission the poll?
Since the AL did not pay for the poll, I accept that to say that the poll was 'commissioned' by the AL is misleading. At the same time it is clear that the poll was done 'for the AL' - and only for the AL - even if, as Radwan says, that it was part of on-going work of an organisation that worked on the poll. The AL was also in control of whether or not to publish the results, and indeed what to publish. Moreover, the AL took part of choosing the particular polling agency.

Q2 - did the AL 'process' the data?
Was the blog wrong in suggesting that the AL processed the data?

He says in his response to me that, 'AL did not do the weighting' and that all of the different lawyers of weighting was done by the INGO, and not by AL. He says that AL 'merely checked the numbers, calculations and formulas used.'

Taking him at his word, the issue is that the independent statisticians did an extra layer of weighting - i.e 'adjust the polls for the respondents' preferences in the last elections' - which AL did not consider appropriate. The difference between the poll published in New Age and Independent is that the New Age contained this extra lawyer of weighting whilst the one published in the Independent did not.

So no processing as such was done by the AL - so the blog appears to be wrong in saying that - however it decided to go ahead and publish a version of the polls different from one that independent statisticians considered a more accurate reflection of the true poll results.

Conclusion
So where does that leave us. I suggest that changes to two paragraphs need to be made - but they are of a relatively semantic nature and do not change the overall thrust of the post.

The paragraph set out above should have stated:

"What both of these articles did not say was the the poll was (a) done for the Awami League with the party's involvement; (b) the raw data was collected by a company owned in part by an Awami League MP (*see ownership details of company at end of article); and (c) the AL's research wing - which is run by Sajeed Wazed, the prime minister's son - decided not to publish the version of the results of the poll suggested by independent statisticians."

One other paragraph also needs to be changed. Later on in the post, it is stated that:

The reason for the difference in the two sets of results is that in those leaked to New Age the raw data had been analysed by independent statisticians - not by the Awami League research team. There is likely to be more integrity to poll results when the raw data is independently analysed.

This needs to be changed to

The reason for the difference in the two sets of results is that the version leaked to New Age included additional weighting that independent statisticians considered should be included, whilst the version leaked to the Independent by the Awami League did not include this additional weighting There is likely to be more integrity to poll results when independent statisticians make these decisions, than those who have an interest in the results.

I have ammended the post as a result. I thanks Radwan for bringing the inaccuracies to my attention.