Surveys are not interviews

The address for the Socialsense bloghttp://culturalscene.blogspot.com.au/
The Thebarton Senior College Moodle

Contact Malcolm at malcolm.mcinerney@thebartonsc.sa.edu.au


Some guidelines and hints on surveys

What is a survey questionnaire?
Survey questionnaires present a set of questions to a subject who with his/her responses will provide data to a researcher.

The design is critical
The key to obtaining good data through a survey is to develop a good survey questionnaire. It is worth spending the time designing your survey to ensure you gain the information you want and that the persons surveyed understand your questions.

Some hints and guidelines with design

* The key to developing a good survey questionnaire is to keep it short while ensuring that you capture all of the information that you need.

* Before you begin to design your survey questionnaire, you developed a set of focus questions for your research. Your survey questions should relate in some way to your focus questions. These questions should serve as a plan for the survey and a question should be rejected if it cannot be seen as helping you answer one of the focus questions in some way.

Types of Questions:

There are two different types of questions that can be used to collect information. The first is called a fixed response question and the second is called an open question. It is important to understand when and how to use these questions when designing your survey.

1. Fixed response
These are questions that offer the respondent a closed set of responses from which to choose. Such a question makes data collection and analysis much simpler and they take less time to answer. Fixed response questions are best suited in the following situations:
• when you have a thorough understanding of the responses so that you can appropriately develop the answer choices
• when you are not trying to capture new ideas or thoughts from the respondent.

Examples of a fixed response question

Do you have a driver's license?
( ) Yes
( ) No
How many hours a day do you spend doing homework?
( ) 0 to 1 hour
( ) 2 to 3 hours
( ) 4 to 5 hours
( ) more than 5 hours

* When writing the selection of responses for a fixed response question, you should make certain that the list covers all possible alternatives that the respondent might select AND that each of the answers is unique (ie they do not overlap).

* For valid data collation it is suggested to use the "Don't know" and other option sparingly.
* You should try to ensure that your respondents are capable of answering the majority of the questions on your survey questionnaire.
* It is suggested that with some fixed choice questions it is better to add a part (b) to the question requesting other choices they wish to make or the opportunity to explain their response.

2. Scaling question (also fixed response but different)

Sometimes you will be interested in obtaining a person's opinion on a topic, subject, product, event, etc.... To capture varying degrees of emotion/opinion about a subject, it is best to use a scale question. A scaling question asks respondents to explain the degree with which they feel about a certain topic, subject, event, etc... For example:

Please describe how you felt about the Harmony day.
(1)Unsatisfied (2)Somewhat (3)Satisfied (4)Satisfied (5)Extremely Satisfied


* In some cases you may wish to have a part b to the scaling question to provide the opportunity for the surveyed to explain why they circled the number they did. Few will take this opportunity but it is worth putting it in your survey.


3. Ranking question (also fixed response but diferent again)

A ranking question asks respondents to explain how they feel about something by comparing it to other items in a list. For example:

Please rank the following Harmony Day activities in order of preference (starting with 1 for your favourite activity).

___ Music
___ Displays
___ Dancing activities
___ Food

* A ranking asks respondents to list their responses in order of preference. This type of question leads you to an answer where the respondent is comparing one thing to another rather than giving you their feeling about each individual item as was the case with the scaling question.

* The disadvantage to a ranking is that if the respondent feels the same about two or more items, they are still forced to sort them into a ranking. The results of a ranking basically tell you which is the most preferred and which is the least preferred item on the list, but you do not know from a ranking if the respondent likes or dislikes any or all of the items on the list.

* It is often a good idea to have part b to a ranking question to provide the respondent with the opportunity to state any other option/s they would have liked to rank.

* Do not have a “other”: category because it makes collation invalid.

4. Open-ended questions

Open-ended questions, are questions where there is no list of answer choices from which to choose. Respondents are simply asked to write their response to a question.

Here is an example:

Example of a open-ended Question
What else would you like to see happen on Harmony Day?

* It is best to use open-ended questions when you are exploring new ideas and you don't really know what to expect from the respondents.

* Open-ended questions let you get more insight into the respondents' thoughts and ideas about a subject.

* The disadvantages to using open-ended questions is that it can be much more time consuming and difficult to analyze the data.

* In general you should try to minimize the number of open-ended questions in your survey questionnaire.

* If you find yourself designing a survey questionnaire where the majority of the questions are open-ended, then you may need to do more exploratory research to get a better foundation of knowledge for the subject you are researching.

General tips to creating a good survey questionnaire:
Here are some tips and tricks to help you ensure you are developing a good survey questionnaire:

* Clearly state your intentions with the research.
At the top of your survey, write a brief statement explaining why you are collecting the information and reassure each respondent that the information is entirely anonymous. If you need to know specifics about a person, respect their privacy by identifying them as subject1, subject2, etc...

* Include instructions with your survey questionnaire
What may seem obvious to you probably is not very obvious to someone else. To ensure that you collect valid survey results, make sure you include instructions on how to answer the survey questionnaire. There should probably be a short introductory set of instructions at the top of the survey questionnaire, and additional instructions for specific questions as needed.
Your overall instructions may be something like:
Please mark the appropriate box next to your answer choice with an "x" ( X ). Please answer all of the questions to the best of your ability.

* Don't ask for personal information unless you need it to help answer your focus questions.

* Keep the questions short and concise
The wording for survey questions should be short and concise. Each question should be clearly stated so that there is no misunderstanding about what is being asked. The best way to ensure your questions are well worded is to test them by having other people review and test your survey before you distribute it to the full sample.
Ask only one question at a time (the double barreled question)
This is a very common mistake in survey questionnaires and one that will severely impact the results of your data. When you are writing a question, you must make sure that you are only asking one question at a time.

Here is an example of a double-barreled question:

How have teachers and students at your school responded
to Harmony Day?
( ) Satisfied
( ) Unsatisfied

It should have been:
How have teachers at your school responded to Harmony Day?
( ) Satisfied
( ) Unsatisfied
How have students at your school responded to Harmony Day?
( ) Satisfied
( ) Unsatisfied

* Make sure the questions are unbiased
When developing your survey questionnaire, you want to make certain that you are asking the questions in a neutral way, ie that you are not leading them toward a particular answer.

* Here is an example of a leading question:

Do you think that the new cafeteria lunch menu offers a better variety
of healthy foods than the old one?
( ) Yes
( ) No
( ) No Opinion

Here is a non-leading question

How do you feel about the new cafeteria lunch menu compared to the old one?
( ) The new menu offers a better variety of healthy foods
( ) The old menu offers a better variety of healthy foods
( ) The selections are similar
( ) No opinion
Ask questions that can be answered by your subjects
Make sure that the questions you are asking are questions that people will be able to answer.

* Order/group questions according to subject
If you have more than six questions in your questionnaire, then you should make an effort to organize your questions so the respondents can answer them as quickly as possible. A good way to organize the questions is to group them together by subject. This way your respondents can focus their thoughts and answer a series of questions around these thoughts. Put the open questions (not part b open questions) at the end of the survey.

* Present the questions in a clean and organized layout
A clean layout will make it much simpler for people to respond to the questions and for you to collect the data. Make sure that your method for marking answers is well explained and that your answer boxes are consistent throughout the questionnaire. Look at this example of a survey.

* Your sample
Make sure that you get a sample of adequate number (at least 20) and one that meets the sample requirements of your research i.e. your survey sample may require a gender balance or gender focus, across ages or of a particular age, across ethnic groups or just one group etc. These sample requirements will depend on the focus of your research. Discuss this sample issue with your teacher.

* Test the survey questionnaire
Once you have developed your survey questionnaire, you should conduct a small test (5 -10 people) to make sure that respondents clearly understand the questions you are asking and that you are capturing the information that you need for your study.

Types of questions for your survey design.

Suggested % in brackets after question type. This is only a guide.
1. Forced choice question:
• Yes/no (15%)
• Selection of choices (15%)

2. Ranking question (20%)
A nominated number of choices (cannot have suggested others – make that part b)

3. Scaled question (20%)
Some form of scale response from 0-10

4. Open-ended response – an open question
• As a question in itself (10%)
• As a follow up open question to either the forced choice, scale or ranking question. The question could be asking for explanation of forced choice, ranking or scale decision or other options that the interviewee would have liked to have as a choice (20%)


The documents to support the devleopment of you survey are on the TSC Moodle for you to download. They are:

* Interview planning document
* Survey planning document
* Sample survey to use as a template

Activity

What do you think?
Have a look at and critique this survey about the 1960 youth. Is it good or bad?