helpsheet WHAT IS RESEARCH? Use this sheet to help you: • understand concepts that are central to research as conducted in the Faculty • improve your ability to conduct effective research 5 minute self test Complete the puzzle by specifying key concepts in research using the hints below. Across 1. A form of research often suited to careful analysis of human behaviours and motivations 2. A statement to be tested in the research process 5. The ability of the research methods to measure what is intended Down 1. A form of research that usually involves application of systematic “scientific” methods 3. The ability of the research methods to produce consistent results 4. An unpleasant condition often experienced by researchers 6. Also known as “information” © The University of Melbourne 2010. These materials were produced by the Teaching and Learning Unit, University of Melbourne. The University of Sydney has reproduced these materials under licence from the University of Melbourne. helpsheet WHAT IS RESEARCH? Introduction This helpsheet outlines the nature of research as it is generally understood in the Business School. Of course, research can be understood in different ways in different disciplines. This helpsheet, therefore, should not be seen as a guide to research in all of its possible forms. The information presented below is adapted mainly from Sekaran (1992), Yin (1994) and Perry (1994), Clarke (2003), and Hurworth, (2003). For further information about the research process see: helpsheets: The Research Process and Research Essentials. See also: helpsheets, Case Studies 1 and Case Studies 2. Quantitative and qualitative research: key differences Academic research is often divided into what is known as “quantitative research” and what is known as “qualitative research”. Key differences between these are summarised below: Quantitative Qualitative Purpose: To determine cause and effect Purpose: To describe on-going processes relationships, i.e., between X and Y. in the real world. Hypotheses: These are usually devel- Hypotheses: These are stated before the oped and refined during the investiga- study and tested during the study. tion. Questions raised determine where the study goes. Theories: These deductively determine Theories: These are developed by induc- the study. tive reasoning. Variables: These are controlled and Variables: No explicit variables. The manipulated to shed light on the hy- focus of qualitative research is to study pothesis. naturally occurring phenomena. Data collection: It is important that data is collected objectively. Data collection: Data can involve subjectivity (e.g., interaction with participants). Design of study: This is stated at the out- Design of study: This can change as the set and does not change. study develops. Page 1 helpsheet WHAT IS RESEARCH? Presentation of data: This is usually numerical (e.g., statistical). Presentation of data: This is usually done in the form of analysis using language (narrative form). Validity and reliability: This is ensured by means of statistical tests. Validity and reliability: This is ensured by means of triangulated (multiple sources) of data/evidence. Data samples: These are carefully chosen to represent larger populations Data samples: Individual cases are studied to shed light on other groups/ cases. Threats to validity: Avoided by means of statistical methods. Threats to validity: Avoided by means of logical analysis to rule out alterative explanations. Subject of analysis: These are simplified and reduced as much as possible. Subject of analysis: These are studied as a whole, as they occur in reality as a complex system. Conclusions: These are stated with statistical measures of confidence (e.g., alpha levels). Conclusions: These are suggestive and always expressed tentatively. From (Clarke, 2003) The differences between quantitative and qualitative research can also be expressed as follows. Quantitative (deductive/linear) Quantitative research typically follows a linear path: one starts with a testable hypothesis, collects data, analyses the data and then accepts or rejects the hypothesis. Hypothesis Data Collection Analysis of results Accept/reject hypothesis ‘The more interest rates rise …’ Qualitative (inductive/spiral) Qualitative research has a very different structure.In the case of qualitative research, the researcher starts with a tentative idea or question, e.g., ‘what is it like working for a major software company in the current economic climate?’ One then observes and asks questions, and analyses what one finds. This guides more specific questioning. Further investigation reveals themes and patterns in the research, which lead to appropriate theoretical study. Finally, one ends with tentative conclusions based on the theoretical insights that one has acquired during the research. In quantitative research, the researcher generally starts with a theoretical statement or position (the hypothesis) and tests this for accuracy. This is therefore deductive research (moving from theory). Page 2 helpsheet WHAT IS RESEARCH? In qualitative research, one starts with observation and ends with a theoretical position or stance. This is, therefore, inductive research (moving to theory). Of course, both kinds of research are not exclusive. Qualitative research may end in a hypothesis that can be quantitatively tested later. Quantitative research may involve qualitative research elements. When is qualitative research needed? As quantitative research is generally well-known, it may be useful to outline when qualitative research is needed. Qualitative research is often appropriate in the following situations: • • • • • • • • When the research is looking at an area that is not well-studied or understood When a subject needs to be studied in a great deal of depth When a holistic perspective is needed When attitudes or behaviours of people need to be studied When measurement techniques like questionnaires are not considered suitable When you are more interested in the process of something (how it works) and not the product (the outcome) When you want to put “content” on statistical results to make the results meaningful When observation of people is considered to be important From (Hurworth, 2003) Case study method uses both qualitative and quantitative research methods. See: helpsheets, Case Studies 1, Case Studies 2 and Case Studies: Research Method Page 3 helpsheet WHAT IS RESEARCH? Qualitative and quantitative data There are several types of data that can be researched. Some of these are listed below: Qualitative Data Utterances: Not just words but complete utterances and the meaning they have in context (e.g., phrases used between workers on the shop floor). Physical actions: For example, what workers do with parts of their bodies when in conversation with fellow workers or their employers; how prospective employees behave in interviews, etc. Quantitative Data Nominal categories: Nominal categories include things like “worker”, “boss”, “employed”, and “unemployed”. Ordinal categories: Ordinal categories are ranked categories such as “strongly agree”, “agree”, “undecided”, “disagree”, etc. Text analysis: This refers, for example, to Interval categories: This refers to quantifi- the reoccurrence of words, their mean- able differences between categories, ing and significance in texts. for example, IQ. Pictures and diagrams: Diagrams and Ratio: These categories can be related pictures can be studied as qualitative as proportions (e.g., income level). data. (Clarke, 2003) Constructs and hypotheses Whether quantitative or qualitative, research must always carried out with a research question or statement (or hypothesis) in mind. It is important to also be sure that the constructs being studied are operational and can be researched successfully. What is a construct? A construct is an abstract entity, not something physical. For example, “employee satisfaction” or “customer satisfaction” are constructs, as is “short term pressure on interest rates”. You must be clear how the constructs in your proposed study can be made operational (objectively testable). You need to ask yourself questions such as these: • ‘How do I decide what “customer satisfaction” means?’ • ‘How can it be measured?’ • ‘What counts as an example of “customer satisfaction”? … and so on.’ Similarly, constructs like: “intelligence”, “company performance”, and “interest”, need to be made operational before you can commence your research project. Page 4 helpsheet WHAT IS RESEARCH? What is a hypothesis? A hypothesis is a statement that relates two or more constructs. For example: The greater the stress experienced in the job, the lower the job satisfaction of employees. Here you need a clear operational definition of “stress” and “job satisfaction”. A good hypothesis is tested by the research that you propose to do. Note that assumptions are not the same as hypotheses. Assumptions are not tested in the research, but hypotheses are. You can also commence and carry out research by taking assumptions to be true, but you cannot do this with the hypothesis(es). In the example above, you might assume the truth of the proposition that stress results in employee dissatisfaction and begin your research. However, you cannot assume the truth of the hypothesis that the greater the stress the lower the job satisfaction. This is, in fact, what you will test. Assumptions in the use of constructs The following assumptions are implicit in the selection of constructs. One must be wary of them and consider them carefully when designing projects (from Clarke, 2003). Selectivity All researchers select what they want to study and leave out other things that they consider irrelevant. However, this selection process is subjective and another researcher may regard something to be relevant that you regard as irrelevant. How would you justify your selection decisions? Indeterminacy Anything that is studied can also be distorted or changed in the process of carrying out a study. A survey, for example, can prompt attitudes that participants initially did not have. Interviews can result in answers that may not have been given outside the interview context. Observers in a company can change the way workers in the company interact. How would you minimise such distortions in your study? Subjectivity All research has a subjective component. How a survey or interview is carried out inevitably reflects the researcher’s views and biases. How would this be minimised? Reductionism Reducing or simplifying a construct can also misrepresent it. For example, you may want to operationalise “customer satisfaction” to be only the number of times a customer returns to the company for further products. But this might result in an inaccurate assessment of satisfaction. Customers might return for other reasons (for example, the company is the nearest one, etc). How would you avoid problems of reductionism in your research? Decontextualisation When something being studied is removed from its usual surrounding context, it can be different or behave differently. Studying something in a laboratory or outside the context of the normal company operations may result in an artificial or inaccurate view of what is really going on. How can you minimise problems of decontextualisation? Page 5 helpsheet WHAT IS RESEARCH? Reification Sometimes the researcher can view something as important when it does not really exist. Constructs can be made “real”, when they are merely the result of the individual researcher being too closely involved in the proposed study and seeing something as “observed”, when it is really only what the researcher wants to see. How would this be avoided in your study? Situatedness There is sometimes a danger of not seeing the observations being made in a wider context of the research activity itself. The researcher is also part of the wider sphere of the experiment, and the research situation is also part of the situation being studied. The aims of the research are also part of the observed aims, and so on. Not being conscious of the situatedness of the research activity can blind the researcher into thinking their work is totally objective and disconnected to what one is studying. How would you be sure that your research does not fall into this trap? Your participation in the research process It is important to recognise the extent to which the researcher is a participant or a nonparticipant in research. Note the typical three types of observer listed below and associated advantages and disadvantages: 1. The uninformed non-participant This is an observer who is not taking part in the particular observed practices being studied and who has not participated in the general practices or setting before. Advantages: Less chance of observer effect and observer bias and a fresh, impartial view on data being collected. However, advantages can be outweighed by the disadvantages: they may not be aware of important information being collected (or they may mistake what is really significant). 2. The informed non-participant This is an observer who has participated before but who is not taking part in the particular practices being studied. Similar advantages to the uninformed non-participant. The disadvantage is that informed non-participants may be biased in the evidence they collect. 3. The informed participant This is an observer who has participated before and is taking part in the practices being studied. Advantages: they can empathise with participants being studied (also a disadvantage). The biggest disadvantages of the informed participant are the potential for observer effect (presence may change data being collected) and observer bias (characteristics of observer may influence what is observed). Finally, those being studied may not behave normally while being observed, especially if they know the observer is among them. Page 6 helpsheet WHAT IS RESEARCH? Validity and reliability In the case of quantitative research in particular, it is important for the indicator used to obtain data in your research to be both valid and reliable. Validity refers to the “extent to which the design of the study, the instrument used to collect the data (e.g., questionnaire) and the results investigate the research questions they were intended to investigate”. (Clarke, 2003). Reliability refers to “the extent to which an investigation produces consistent results” (Clarke, 2003). The following table summarises the various kinds of validity and reliability issues that need to be considered. (Clarke, 2003; Neuman, 1994). Validity Reliability Face validity: The indicator (survey, questionnaire) really does measure the construct under examination. Stability reliability: This is reliability over time (test and retest). Content validity: The indicator measures all aspects of the construct and not just a part. Representative reliability: This is reliability over sub-populations (men and women). Criterion validity: The indicator corresponds with and is predictive of measurements using related indicators. Equivalence reliability: This is reliability across different indicators used (e.g., questionnaires and inter-rater reliability measurements). Construct validity: The indicator measures the construct in a manner which is convergent with other measures in terms of direction (e.g., the level of education and income level converge). The indicator also allows discrimination of opposing constructs. Internal validity: The indicator and research design chosen should not allow different interpretations of the data. Internal validity should be “high”. External validity: Results gained from the indicator and research design should be generalisable to other cases that are not being studied. External validity should be “high” Statistical validity: The indicator being used should allow data to be measured using the most suitable statistical instrument. Consequential validity: Results from the study should be appropriate and relevant to the research context. Page 7 helpsheet WHAT IS RESEARCH? The Hypothetico-deductive method This research method should not be seen as strange and mysterious. It is a natural combination of the emphasis of both qualitative and quantitative research methods. It may turn out that the research you are required to produce (in terms of written work) in your department is strictly quantitative or qualitative. However, in practice, your research will generally progress through the stages given above. Page 8 helpsheet WHAT IS RESEARCH? Answers 5 minute self test Across 1. Qualitative 2. Hypothesis 5. Validity Down 1. Quantitative 3. Reliability 4. Stress 6. Data References Chalmers, A. (1982). What is This Thing Called Science? St. Lucia: University of Queensland Press. Clarke, D. (2003). Research Methods in Education. Unpublished manuscript, Melbourne. Hurworth, R. (2003). Overview of Qualitative Methods. Unpublished manuscript, Melbourne. Neuman, W. L. (1994). Social Research Methods (3rd ed.). Boston: Allyn and Bacon. Perry, C. (1994). A Structured Approach to Presenting PhD Theses: Notes for Candidates and their Supervisors. Paper presented at the ANZ Doctoral Consortium, Sydney University. Sekaran, U. (1992). Research Methods for Business: A Skills-Building Approach. New York: Wiley and Sons. Yin, R. (1994). Case Study Research: Design and Methods (Vol. 5). London: Sage. Page 9