Firm behind 'essential' workers study shares research process openly

Market research company Milieu Insight has offered full transparency around the survey methodology and design regarding the recent infographic about essential workers published on The Sunday Times.

Based off data provided by Milieu Insight, the infographic said that artists were deemed as the top five non-essential workers, followed by telemarketers, social media manager/PR specialists, business consultants and human resource managers. This sparked criticism from those helming “non-essential” jobs, while some industry players in the world of marketing and communications took to LinkedIn to express their displeasure.

Milieu COO Stephen Tracy said in a recent blog post that as comprehensive details about the methodology and survey design are not freely available to everyone as it is hosted behind a paywall, the firm feels it is important to offer information about the background of the study. Tracy explained details about its panel methodology, the specifications for The Sunday Times poll, and some additional information about the exact questions that were asked. Milieu manages an online access panel (also known as non-probability sampling), where it recruits participants to join its opinion sharing community.

The poll, commissioned by The Straits Times on 3 June, had an initial set of questions provided by The Straits Times’ editorial team. A review of the survey was conducted by the Milieu team, and revisions were proposed, such as changes to question types and survey flow. Tracy explained that the questions in the survey addressed current perceptions and attitudes toward low-wage essential workers in Singapore, and whether there is support for paying them more.

However, given much of the debated focus has been on the data points related to essential versus non-essential jobs, he broke down how respondents were prompted with the following definition at the beginning of the survey.

Respondents for the survey were told: "In this survey we are going to be asking your opinion about essential workers in Singapore. By 'essential workers' we mean someone who is engaged in work deemed necessary to meet basic needs of human survival and well-being, such as food, health, safety and cleaning."

This statement was included to ensure there was a baseline understanding of the term “essential workers” among the respondents from the beginning of the survey. Next, respondents were shown two questions:

Q1. Please select the jobs that you think are MOST essential from the list below.
Q2. Now select which of the remaining jobs you think are absolutely NOT essential.

The list below was then displayed to respondents for both questions. However, for Q2, only options that they did not select in Q1 were displayed (a mechanic known as negative response piping). The response list was also randomised, meaning each respondent saw a different ordering of the jobs list:

• Artist
• Cleaner
• Construction worker
• Business consultant
• Corporate lawyer
• Deliveryman
• Doctor
• Engineer
• Garbage collector
• Hawker
• Human resource manager
• IT technician
• Nurse
• Politician
• PR specialist
• Security guard
• Social media manager
• Teacher
• Telemarketer
• University professor
• I don't think any of these are essential / non-essential

Methodology: Online (Milieu panel)
Sample size: n=1,000
Target audience: Singapore adult population (residents and citizens), aged 16 and above
Field work dates: June 12th to 15th, 2020
Survey duration: 19 questions (approx 5 mins)
Margin of error: +/- 3% at a 95% confidence level
Note: At n=1,000 samples (with sample balancing in place) the margin of error for this study is +/- 3%. What does this mean? Let’s take this statistic as an example - 73% of the respondents indicated that “I respect essential workers more now”. With a margin of error of +/- 3% this means that the researchers are 95% confident that between 70% to 76% of the population would answer this way if the entire adult population of Singapore was interviewed. 

To achieve representative results, Milieu used a technique known as sample balancing where quotas are  applied for certain demographic variables that reflect the current national statistics. In this case, the poll used interlocking quotas of age and gender to achieve a representative sample of Singaporean adults, aged 16 and above.

Beyond sample balancing, Milieu also applied a wide range of fraud detection and quality control mechanisms at both the panel and survey level. This includes a wide range of automated attention checks to ensure respondents are both paying attention and providing Milieu with consistent responses throughout a survey. As such, The Sunday Times poll was a sample size of 1,000 adults aged 16 and above. A set of 19 questions were asked, and the polls were collected between 12 to 15 June. He added:  

Respondents were not asked to rank the jobs/professions, rather, they selected the ones they deemed essential versus not essential given the definition provided.

Why was PR and social media deemed non-essential?

In a conversation with Marketing, Tracy added that for those working in professions such as PR and social media marketing, it’s important to be clear that the results of the poll do not conclude Singaporeans or society at large don’t value your profession, industry and hard work.

“I’ve worked in PR myself, and Milieu happen to employ people in three of five professions the Sunday Times poll lists as "non-essential". So this debate hits close to home. Ironically enough, we don’t have a PR division or agency yet (but I think we’re considering getting one!). But from my own career experience, I know firsthand the immense value of good marketing and communications,” he added.

“Well executed comms are what brought Samsung back from what could have been a product ending crisis with the Samsung Note 7 exploding phones. After the firm ran full-page editorials in major newspapers like the NYTimes apologising for the incident and committing to a full investigation, the Note 8 went on to be Samsung’s most successful phone launch. And that was a direct result of well crafted, strategic brand communications,” he added.

So do Singaporean's think professions like PR and social media marketing are not essential? In the context of survival and meeting basic human needs, maybe, he added.

“I think that definition, which was shown to the survey respondents, should have been included in The Sunday Times infographic and article.  But in a different context, like are these professions essential to our economy, or to building progressive and successful companies, of course they’re essential! I don’t think I’d want to live in a world without great storytelling, and I think we can all agree to that,” he added.

Do Singaporeans think artists and professions such as HR are non-essential?

On whether Singaporeans think artists and professions such as HR are non-essential, Tracy said the simple answer is, not necessarily. "Again, respondents answered according to the definition of ‘essential workers’ that was shown at the beginning of the survey, which was clear about meeting ‘basic human needs’. Some could argue that art is a basic human need, and many of us at Milieu would agree,” he said.

However, in the context of a post-COVID-19 world, and based on the definition of essential workers provided, these are the jobs, based on a shortlist of professions, that Singaporean adults deem essential versus non-essential.

Nonetheless, Tracy was quick to add that does not mean that Singaporeans view professions such as artists or HR professionals as not valuable or valued, or even essential in the context of something else, such as happiness or stability. Agreeing that these are important questions to be asked, and perhaps can be done in another study, he added that Milieu is confident if asked, most Singaporeans would agree that artists and their work, be it a musician, painter, dancer or filmmaker, are absolutely essential to the enjoyability of everyone’s lives and own mental health and well-being.

Tracy also added that many Singaporean working professionals would likely also agree that HR is the backbone of most successful, productive, and progressive companies today. “There are indeed questions that the survey doesn’t answer, as well as some caveats for how we interpret the results of the questions that were asked. Either way, we hope that this offers greater transparency around the methodology, as well as some added perspective on the implications of some of the findings,” he explained.

Related articles:
Analysis: Are your PR and social jobs essential?