Please note that two paragraphs of this post were amended, following a request for a correction. You can read about this in a separate post here. The two corrected paragraphs can be identified by *
---------------------------
Last Sunday, Bangladesh's English language Independent newspaper carried on its front page, a story under the headline, 'AL takes lead over BNP' which stated that the Awami league had a lead over the BNP of 6 percentage points. The first sentence read: 'With the support of 39.6 per cent of the electorate, the ruling Awami League has a clear edge in the election race over opposition BNP, which is favoured by 34.2 per cent voters, shows a mid-October poll.'
A similar story was also published in the Bengali language daily Ittefaq a few days later
What both of these articles did not say was the the poll was (a) done for the Awami League with the party's involvement; (b) the raw data was collected by a company owned in part by an Awami League MP (*see ownership details of company at end of article); and (c) the AL's research wing - which is run by Sajeed Wazed, the prime minister's son - decided not to publish the version of the results of the poll suggested by independent statisticians.* (This para was changed on 23 Dec)
In themselves, of course, none of the above should necessarily mean that the poll is not trustworthy.
Polls commissioned by parties, even when done by polling firms with a possible political bias, and analysed by party researchers themselves, can in principle be done with professionalism and integrity.
However, this scenario does create a perception of bias, particularly when of course all other independent polls had shown BNP leads of between 11 and 15 percent. To move from such a significant BNP lead to a 6 percent AL lead in such a short period of time (the last poll being done in September) seems somewhat unlikely.
There are two issues here; first the failure of the newspaper to mention any of the information set out above and secondly, how one should view such internal party polls.
On the first issue, clearly the paper should have provided this information to its readers in order to allow them to make up their own minds about the likely integrity of the poll. It would seem fair to assume that the information about the poll was left out in order to give the impression that it was entirely independent. The results may be right, but the poll was by no means independent!
It should be no surprise perhaps to anyone that it was Bangladesh's Independent Newspaper and Ittefaq that carried these stories in this way. The Independent is owned by Salman Rahman, an advisor to the prime minister, and Ittefaq is owned by Anwar Hossain Monju, a minister under a previous Awami League government, whose Jatiya (Monju) party early on agreed to take part in the upcoming elections. These were safe newpapers to whom the Awami League could give the poll and who would either not ask any obvious questions or who were happy to publish them in the way that they did though knowing about the provenance of the poll.
On the second issue on how one should view internal party polls, let me quote from the current doyen of US poll analysts Nate Silver who shot to fame when he accurately predicted the results in all 50 states in the 2012 US general election, having predicted 49 of the 50 states in the election before that. In an article written on his New York Times blog after the last election he wrote that internal party political polls could not be relied upon. He said
Interestingly, around the time the Independent published its polls, a different version of the results of this same poll was leaked to New Age.
Unlike the results published by the Independent and Ittefaq, this showed that instead of the AL being in the lead, the BNP continued to have a 3 percent lead over the Awami League. Clearly an apparent big decline in support for the opposition party, but no where near as disastrous a set of results as the Independent/ Ittefaq published results suggested.
The reason for the difference in the two sets of results is that the version leaked to New Age included additional weighting that independent statisticians considered should be included, whilst the version leaked to the Independent by the Awami League did not include this additional weighting There is likely to be more integrity to poll results when independent statisticians make these decisions, than those who have an interest in the results.* (This para was changed on 23 Dec)
Of course, questions might still be raised about the mechanics of the poll - that is to say the integrity of the collection of the raw data from a randomized selection of people (particularly as the polling firm has a possible Awami League bias) - though my understanding is that this was done in an appropriate manner.
Going back to the articles published in the Independent/Ittefaq, it is also notable that the part of the polls that was not released to the paper - or the papers did not publish - was the most damaging part to the Awami League right now: the significant support that people have for elections under a caretaker government.
So where does this all leave us. To some extent even talking about opinion polls when it looks like the BNP will boycott the forthcoming elections appears at odds with reality. But it is still important to assess the support held by different parties. A boycotted election by the BNP which the party could argue it would more than likely win if the election was undertaken at the time of a 'neutral government' will be viewed differently than if the boycott took place at the time when the BNP was more likely to lose were such a vote to take place in such a situation. Optics matter.
So, has there been a significant swing towards the AL, is the Awami Leage poll an outlier poll, or is it just a biased one?
As Nate Silver has said, polls change in response to news, they don't tend to change for no good reason.
Has there been any thing significant to have happened between September when Prothom Alo did its poll and found the BNP at 50 percent and when the Awami League did it's poll which found, the BNP at 38 percent - a swing of 12 percent? It does not seem so.
Of course the swing between the Nielsen/Democracy International poll in July which found BNP's support at 43 percent and the Awami League's poll is only 5 percent, which seems much more possible.
Perhaps the most interesting thing about the polls is that there remains a significant percentage of people who have not made up their minds, many of whom will be waiting until the last moment to decide which party to vote for (assuming there is a participatory election)
The problem in Bangladesh is that not enough trusted independent professional polls are done - and it is therefore very difficult to quite know how to judge these scarce polls. However, I think it is probably fair to conclude that BNP's lead is slipping somewhat - indeed New Age's analysis of Nielsen/Democracy International's poll did suggest that there were reasons for the AL to think that BNP's lead was weak.
More independent polls should be commissioned, independent statisticians should analyse them and results along with the raw data published.
-----
* According to records in the Register of Company, MRC Mode Ltd is owned by Market Modeliers (India) Pvt Ltd (400 shares); Mrs Sara Zaker (311 shares); Aly Zaker (200 shares); Asaduzzaman Noor (200 shares). The three named individuals are also directors, with Sara Zaker being its Managing Director. The Indian company has two individuals representing it on the board. The company was originally registered in 1991.
---------------------------
Last Sunday, Bangladesh's English language Independent newspaper carried on its front page, a story under the headline, 'AL takes lead over BNP' which stated that the Awami league had a lead over the BNP of 6 percentage points. The first sentence read: 'With the support of 39.6 per cent of the electorate, the ruling Awami League has a clear edge in the election race over opposition BNP, which is favoured by 34.2 per cent voters, shows a mid-October poll.'
A similar story was also published in the Bengali language daily Ittefaq a few days later
What both of these articles did not say was the the poll was (a) done for the Awami League with the party's involvement; (b) the raw data was collected by a company owned in part by an Awami League MP (*see ownership details of company at end of article); and (c) the AL's research wing - which is run by Sajeed Wazed, the prime minister's son - decided not to publish the version of the results of the poll suggested by independent statisticians.* (This para was changed on 23 Dec)
In themselves, of course, none of the above should necessarily mean that the poll is not trustworthy.
Polls commissioned by parties, even when done by polling firms with a possible political bias, and analysed by party researchers themselves, can in principle be done with professionalism and integrity.
However, this scenario does create a perception of bias, particularly when of course all other independent polls had shown BNP leads of between 11 and 15 percent. To move from such a significant BNP lead to a 6 percent AL lead in such a short period of time (the last poll being done in September) seems somewhat unlikely.
There are two issues here; first the failure of the newspaper to mention any of the information set out above and secondly, how one should view such internal party polls.
On the first issue, clearly the paper should have provided this information to its readers in order to allow them to make up their own minds about the likely integrity of the poll. It would seem fair to assume that the information about the poll was left out in order to give the impression that it was entirely independent. The results may be right, but the poll was by no means independent!
It should be no surprise perhaps to anyone that it was Bangladesh's Independent Newspaper and Ittefaq that carried these stories in this way. The Independent is owned by Salman Rahman, an advisor to the prime minister, and Ittefaq is owned by Anwar Hossain Monju, a minister under a previous Awami League government, whose Jatiya (Monju) party early on agreed to take part in the upcoming elections. These were safe newpapers to whom the Awami League could give the poll and who would either not ask any obvious questions or who were happy to publish them in the way that they did though knowing about the provenance of the poll.
On the second issue on how one should view internal party polls, let me quote from the current doyen of US poll analysts Nate Silver who shot to fame when he accurately predicted the results in all 50 states in the 2012 US general election, having predicted 49 of the 50 states in the election before that. In an article written on his New York Times blog after the last election he wrote that internal party political polls could not be relied upon. He said
My database of campaign polls released to the public in United States House races found that they were about six points more favorable to their candidate than independent surveys on average — and that they were typically less accurate in the end.
The traditional explanation for this phenomenon is that the subset of campaign polls that are released to the public is subject to a type of selection bias. Campaigns conduct polls all the time, but only occasionally disclose these results to the public and will be much more inclined to do so when the numbers are favorable for their candidates (especially in comparison to independent polls). In essence, the internal polls that filter their way into the public domain may be the outliers. ...
But when campaigns release internal polls to the public, their goal is usually not to provide the most accurate information. Instead, they are most likely trying to create a favorable news narrative — and they may fiddle with these assumptions until they get the desired result. (emphasis added)
Some reporters make the mistake of assuming that information is valuable simply because it is private or proprietary. But the information that makes it to the reporter’s ears, or into his in-box, may be something that the campaign wants him to hear or see.
The release of internal campaign polls is not that different from any other form of spin in this respect — but that is precisely the point. Internal numbers that a campaign releases to the public should be thought of less as scientific surveys and more as talking points.Of course the US elections are in so many way not comparable to those in Bangladesh - for one thing, there are far fewer polls conducted here. But nonetheless Silver's point about the the integrity/bias of internal political polls and the reason why parties may release the results is a salient point to consider.
Interestingly, around the time the Independent published its polls, a different version of the results of this same poll was leaked to New Age.
Unlike the results published by the Independent and Ittefaq, this showed that instead of the AL being in the lead, the BNP continued to have a 3 percent lead over the Awami League. Clearly an apparent big decline in support for the opposition party, but no where near as disastrous a set of results as the Independent/ Ittefaq published results suggested.
The reason for the difference in the two sets of results is that the version leaked to New Age included additional weighting that independent statisticians considered should be included, whilst the version leaked to the Independent by the Awami League did not include this additional weighting There is likely to be more integrity to poll results when independent statisticians make these decisions, than those who have an interest in the results.* (This para was changed on 23 Dec)
Of course, questions might still be raised about the mechanics of the poll - that is to say the integrity of the collection of the raw data from a randomized selection of people (particularly as the polling firm has a possible Awami League bias) - though my understanding is that this was done in an appropriate manner.
Going back to the articles published in the Independent/Ittefaq, it is also notable that the part of the polls that was not released to the paper - or the papers did not publish - was the most damaging part to the Awami League right now: the significant support that people have for elections under a caretaker government.
So where does this all leave us. To some extent even talking about opinion polls when it looks like the BNP will boycott the forthcoming elections appears at odds with reality. But it is still important to assess the support held by different parties. A boycotted election by the BNP which the party could argue it would more than likely win if the election was undertaken at the time of a 'neutral government' will be viewed differently than if the boycott took place at the time when the BNP was more likely to lose were such a vote to take place in such a situation. Optics matter.
So, has there been a significant swing towards the AL, is the Awami Leage poll an outlier poll, or is it just a biased one?
As Nate Silver has said, polls change in response to news, they don't tend to change for no good reason.
Has there been any thing significant to have happened between September when Prothom Alo did its poll and found the BNP at 50 percent and when the Awami League did it's poll which found, the BNP at 38 percent - a swing of 12 percent? It does not seem so.
Of course the swing between the Nielsen/Democracy International poll in July which found BNP's support at 43 percent and the Awami League's poll is only 5 percent, which seems much more possible.
Perhaps the most interesting thing about the polls is that there remains a significant percentage of people who have not made up their minds, many of whom will be waiting until the last moment to decide which party to vote for (assuming there is a participatory election)
The problem in Bangladesh is that not enough trusted independent professional polls are done - and it is therefore very difficult to quite know how to judge these scarce polls. However, I think it is probably fair to conclude that BNP's lead is slipping somewhat - indeed New Age's analysis of Nielsen/Democracy International's poll did suggest that there were reasons for the AL to think that BNP's lead was weak.
More independent polls should be commissioned, independent statisticians should analyse them and results along with the raw data published.
-----
* According to records in the Register of Company, MRC Mode Ltd is owned by Market Modeliers (India) Pvt Ltd (400 shares); Mrs Sara Zaker (311 shares); Aly Zaker (200 shares); Asaduzzaman Noor (200 shares). The three named individuals are also directors, with Sara Zaker being its Managing Director. The Indian company has two individuals representing it on the board. The company was originally registered in 1991.
No comments:
Post a Comment