Research Methods and Analysis
Qualitative and quantitative methods, the full range of data-collection techniques from observation to content analysis, and the toolkit of variables, sampling, hypothesis, reliability and validity.
Contents
- Qualitative and quantitative methods
- Techniques of data collection
- Other methods
- Naturalistic research
- Documentary method
- Variables, sampling, hypothesis, reliability and validity
Qualitative and quantitative methods
Quantitative sociology is generally a numerical approach to understanding human behaviour.
Qualitative sociology opts for depth over breadth. Describes reality as experienced by the groups, communities and individuals.
Features of quantitative research
| Social facts | Statistical data | correlation | causation | Generalization and replicability |
|---|
Features of qualitative research
| Emphatic description of social reality | Contextualism | Emphasis on processual dimension | flexibility |
|---|
| Quantitative | Qualitative |
|---|---|
| Problem is specific and precise | General and loosely structured |
| Hypothesis formulated before the study | During or after |
| Concepts are operationalised | Sensitized |
| Research design is prescriptive | Not prescriptive |
| Sampling planned before data collection | During |
| Sampling is representative | Not representative |
| All types of measurement scales are employed | Nominal scales are used |
| Inductive generalization is made | Analytical generalization |
| Findings are highly integrated | Mostly not integrated |
Techniques of data collection
Observation
Observation entails gathering data through vision as its main source. Lindsey Gardner has defined observation as selection, provocation and encoding of that set of behaviours and settings concerning organism’s naturalistic settings and familiar surroundings which are consistent with empirical aims.
Participant observation: Researcher joins the group they intend to study and observe it from inside.
Example: Nels Anderson – Hobo tribes. Verier Elwin – married a Gond woman.
Uncontrolled participant observation: Control refers to the standardization of observational techniques or in some cases control over the variables in an experimental situation.
Advantages of participant observation
| Behaviour least affected | Able to record natural behaviour | Access to information not easily available |
|---|---|---|
| Able to record context | Material collected of wider range | Can check the truthfulness of statement |
| More difficult for people being studied to lie | Date is more valid |
Disadvantages of participant observation
| Range of experience narrowed | Researcher can affect the group behaviour |
|---|---|
| Ethical issues – deceiving those observed | Researcher may lose objectivity |
| Group behaviour may seem natural to observer | Interprets subjectively |
| Fails to specify procedure – difficult to replicate | Methods cannot be used for studying illegal activities |
Non-participant observation: Investigators study their subject from outside. Purely non-participant observation is difficult.
Advantages of observation (Bailey)
| Superior in data collection on non-verbal behaviour | Intimate informal relationship |
|---|---|
| Natural environment | Longitudinal analysis |
Disadvantages of observation – (Bailey)
| Lack of control | Quantification difficulties | Small sample size | Difficulty in entry | Limited study |
|---|
Williamson et al
| Not applicable to large social settings | Only few safeguards against biases of researcher | |
|---|---|---|
| Selectivity problem in data collection | Researcher presence may change group | Difficult to replicate |
Other types of observation
-
Systematic observation – explicit procedure is used in observation and recording by following certain rules.
-
Unsystematic observation – does not follow any rule or logic
-
Naïve observation – unstructured and unplanned
-
Scientific observation – systematically planned and executed, related to a goal and subjected to tests and control.
-
Structured observation – formal procedure and a set of well defined observation categories
-
Unstructured observation – loosely organized
-
Natural – observations made in natural settings
-
Observation is made in laboratory
-
Direct observation – observer plays a passive role
-
Indirect observation – observer observes the physical traces which phenomenon under study has left behind
-
Covert observation – subjects are unaware that they are being studied
-
Overt observation – subjects are aware that they are being studied
Process of observation
| Selection of topic | Formulation of topic | Research design |
|---|---|---|
| Collection of data | Analysis of data | Report writing |
| Structured observation | Unstructured observation |
|---|---|
| Strict design | Flexible |
| Non-participant observation | Participant observation |
| High standardization | Low |
| Laboratory or natural settings | Natural |
| Studies small group | Small and large group |
| Formal observation | Informal |
| Unobtrusive observation | Obtrusive |
| Direct | Indirect |
| Employs quantitative design | Qualitative |
Interview
Interview is verbal questioning. Consists of researcher asking the interviewee a series of questions.
Types of interviews
-
Structured interview - wordings of the questions and the order in which they are asked remains the same in every case.
-
Unstructured interview – no specifications in wording and order of the questions.
-
Standardized interview – answer to each question is standardized as it is determined by a set of response category.
-
Unstandardized – responses are left open to the respondents
-
Individual interview – interviewer interviews only 1 respondent
-
Group interview – more than 1 respondents are interviewed simultaneously
-
Self – administered interview - respondent is provided with questions and instructions of writing the answer
-
Other administered – interviewer writes answers on the response sheet
-
Unique interview – interviewer collects entire information in 1 interview
-
Panel interview – interviewer collects information from the same group of respondents 2 or more times at regular intervals
-
Personal interview – face to face contact between the interviewer and the interviewee
-
Non-personal interview – no face to face contact
Advantages of interview
| Non-verbal behaviour can be observed | Identity of respondent is known | Administration is easy |
|---|---|---|
| Interviewer can explain difficult terms | Completeness of interview | Response rate is high |
| Respondents confidence can be sort through personal rapport | In-depth probing possible |
Disadvantages of interview
| Hide or give wrong information due to identity fear | More costly/ time consuming than questionnaire |
|---|---|
| Response depends upon interviewee’s mood | Variability of responses with different interviewers |
| Interviewer may respond differently based on his interpretations | |
| Less anonymity | Less effective for sensitive questions |
Questionnaire
Questionnaire is described as a document that contains a set of questions, the answers to which are to be provided personally by the respondents.
Types of questions
-
Primary – elicit information related to the research topic
-
Secondary – elicit information not relate directly related to the topic
-
Tertiary – to establish a framework for convenient data collection
-
Open ended questions – allows the respondents to compose his own answer rather than choosing from a number of given answers.
-
Closed or fixed choice questions – requires a choice between a number of given answers.
-
Direct question - personal question which elicit information about the respondent
-
Indirect question – seeks information about other people
Advantages of open ended questions
| Researcher gets insight | Respondent gets freedom | Varied information | Preferable for complex issues |
|---|
Disadvantages of open ended questions
| Response maybe irrelevant | Statistical analysis difficult | Difficult to classify and code responses |
|---|---|---|
| Lengthy responses leads to time consuming analysis | Difficult for semi-literate respondents |
Advantages of close ended questions
| Greater uniformity of responses | Easy to code, score and process | Answers can be compared |
|---|---|---|
| Less time to complete questionnaire | Respondents clear about the meaning of the question | |
| Irrelevant responses are not received | High response rates particularly in sensitive questions |
Disadvantages of close ended questions
-
Some responses might be omitted
-
Respondent may not be able to relate to the options provided
-
Detecting whether the respondent has ticked the right answer is not possible
Steps in questionnaire preparation
| preparation | First draft | Self evaluation | External evaluation |
|---|---|---|---|
| revision | Pre test or pilot study | Revision & second pre test | Final draft |
Guidelines for framing questions
| Clear & unambiguous | Relevant | Short | Negative questions should be avoided |
|---|---|---|---|
| Biased terms should be avoided | Will the respondents be competent and willing to answer? |
Advantages of questionnaire
| Lower cost | Time saving | Accessibility to widespread respondents | No interviewer’s bias |
|---|---|---|---|
| Greater anonymity | No variations | Respondents convenience | Standardized wordings |
Disadvantages of questionnaire
| Mailed questionnaires can be used only for educated people | Mailing address may not be correct | |
|---|---|---|
| Misunderstanding of questions cannot be corrected | Bias in response selectivity | |
| No opportunity to collect additional data | Return rate is low | Many questions remain unanswered |
| No proof as to who has answered the question | Lack of depth of probing | |
| Respondents can consult others hence cannot be considered as respondent’s opinion | ||
| Reliability of respondent’s background cannot be verified | ||
| Size of questionnaire has to be kept small so full information cannot be obtained |
Other methods
Experiments
Experiment establishes the relationship between independent variables and ensures it remains intact and free of distortions. It entails Max-con-Min
Experimental sampling:
Consists of 2 study groups;
-
experimental - exposed to the independent variable
-
Control group – not exposed to the independent variable.
Basic steps of data collection
-
Pre-test
-
Test
-
Post test
Types of experiments
-
Laboratory experiment
-
Field experiment
-
True field experiment
-
Quasi field experiment
-
Demonstration experiments
Experimental design
-
Before-after design
-
Classical experimental design
-
After only design
-
Solomon two control group design
-
Randomized group design
-
Solomon four group design
Limitations of experiments
-
Maturation – change in dependent variable due to maturation of the subjects
-
Conditioning – subjects may develop interest in the experiment and respond atypically
-
History effect – historical events might occur between pre & post test & affect the responses
-
Changes in samples – Due to mortality, spatial mobility or gender unavailability
-
Instrumentation – changes in dependent variable due to the changes in the nature of the test
-
Interaction
-
Sampling – changes in the dependent variable due to sampling problems
-
The Hawthorne effect – changes due to the fact that subject knows they are being studied
-
Modeling – the subjects may wish to please the experimenter
-
Ecology
Panel studies
Panel studies are a particular design of longitudinal study in which the unit of analysis is followed at specified interval over a long period, after many years.
Characteristics of panel studies
-
They study the same sample on more than one occasion.
-
They study the same topic.
-
They employ the same method.
Factors affecting quality of panel study – limitation
| Loss of subjects | Conditioning | Instrument bias | Respondent bias | Study conditions | Costs |
|---|
Focus groups
It is a loosely constructed discussion with a group of people brought together for the purpose of the study, guided by the researcher and addressed as group.
Weakness of focus group
-
Being in group, participants may hide their real opinions.
-
Recording can be problematic
-
Domination by some members
-
Some members may not participate
-
Attempts to agree with the leader
-
Difficulties of keeping discussion on track
-
Group members may offer a collective opinion
-
Findings may not be representative
Naturalistic research
Field research
It is a systematic study of ordinary events and activities as they occur in real-life situations
Types of field research
-
Particularistic: Focuses on social issues and situations aiming to understand their structures, processes and outcomes as they occur and as displayed in the behaviour of those involved in the study, but without reference to overarching context such as culture.
-
Holistic field research: Focuses on culture as whole entities, their structure and characteristics per se as well as in comparison with other cultures.
Ethnographic research
It is the science of ethnos, i.e. people or culture. It is the science of cultural description and interpretation of a cultural or social group system, the study of cultures with purpose of understanding them from the native point of view.
Features of ethnographic research
-
Form of field research that studies culture
-
Employs holistic approach
-
Conducted in natural settings
-
Understands culture from within
-
Constructs culture through in-depth studies
-
Aims at analysis of life-worlds
-
Stresses on subjectivity
Advantages / strengths of ethnographic studies
-
Holistic perspectives
-
High degree of flexibility
-
Capacity to identify contradiction and consistencies
-
High quality researcher-participant relationship
-
Closeness of the participants
-
High external validity
-
High sensitivity to subtle nuances of meaning and significance
-
Capacity for longitudinal study
Disadvantages / weaknesses of ethnographic studies
-
Inability to provide evidence supporting causality
-
Inability to ensure reliability and validity
-
Lack of application
-
Inability to ensure objectivity
-
Difficulty with going native
-
Distortion of the natural setting by the very presence of the observer
Case studies
A case study is an empirical inquiry that investigates a contemporary phenomenon within a real life context when the boundaries between the phenomenon and context are not clearly evident and in which multiple sources of evidence are used.
Types of case studies
-
Intrinsic case studies: Conducted for its own sake to learn about the case only.
-
Instrumental case studies: To inquire into a social issue or to refine a theory
-
Collective case studies: Includes a number of single case studies investigated jointly for the purpose of inquiring into an issue, phenomenon, group or condition. It normally includes several instrumental studies.
Characteristics of case studies
-
conducted in natural settings
-
suitable for pursuing depth analysis
-
studies whole units
-
entails a single or a few cases only
-
it studies typical cases
-
it perceives respondents as experts and not sources of data
-
employs many and diverse methods
-
employs several sources of information
Advantages / strength of case studies
-
Allows in-depth research
-
Produces first hand information
-
Employ methods that encourage familiarity and close contact with informants
-
Allows the employment of variety of interrelated methods and sources
-
Focuses on direct and verifiable life experiences
-
Produces information that covers the whole unit
Disadvantages / weaknesses of case studies
-
Results relate to the unit analysis only and allow no inductive generalization
-
Findings entail personal impressions & biasness hence no assurance of objectivity, validity & reliability
-
Research cannot be replicated
-
The interviewer effect may cause distortions
-
Time consuming
Documentary method
Document study
The focus of document analysis is on description, identification of trends, frequencies, interrelationships and sometimes statistical analysis.
Types of approaches
-
Descriptive analysis – elementary and entails summarizing data
-
Categorical analysis – based on categories constructed before the commencement of the studies
-
Exploratory analysis – searches for peculiarity, characteristic attributes and trends in the text.
-
Comparative analysis – compares social issues across time, countries and cultures.
Biographical research
Refers to the study of personal and biographical documents that intentionally or unintentionally offer information about the structure, dynamics and function of the author’s consciousness.
Relevant documents are diaries, memoirs, autobiographies, letters, witness statements.
The method focuses on 2 points;
-
The authors definition of self and social action
-
The social regulation of individuality
Secondary analysis
Deals with data gathered by researchers, public institutions or government authorities.
Criteria of secondary analysis:
-
Concerned with analyzing already collected data
-
Implies that the previously collected data were fully processed and analyzed
-
Produces new and more detailed information and different conclusion.
-
Addresses aspects of the issue that are different from those of the original author
-
Employs sophisticated methods
-
Employs quantitative or qualitative method
-
Focuses on private or official source of data
-
Secondary because it analysis the data for the second time
Meta analysis
It converts the result of a number of studies to a common measure so that their findings can be compared and integrated into a common conclusion.
Advantages / strengths of document studies
| Enables retrospective study | Quick & easy access | spontaneity | Convenience | Possibility of retesting |
|---|---|---|---|---|
| High quality information | Less time consuming | Sole source | Low cost | Non-reactivity |
Disadvantages / weaknesses of document studies
| Lack of representativeness | Lack of accessibility | Incomplete data | Personal bias |
|---|---|---|---|
| Questionable reliability of some docs | Comparisons not always possible | Methodological problems |
Content analysis
Content analysis is a documentary method that aims at a quantitative and / or qualitative analysis of the content of the texts, pictures, films and other forms of verbal, visual or written communication.
Major types of content analysis
-
Descriptive content analysis – aims at identifying and describing the main content of data
-
Contextual analysis - aims to understand the context through the meaningful statement of the authors.
-
Comparative content analysis – entails comparing texts of different media / authors to identify ideological or other differences.
-
Processual or particularistic content analysis –studies elements or aspects of whole process.
Advantages / strength of content analysis
-
Unobtrusive
-
Can be used when access to research topics or units is not possible
-
Does not involve respondents so avoids problems associated with them
-
Eliminates researcher bias as it involves already complete material
-
Accessibility of the research material
-
Low cost method
-
Less bias than other methods
Disadvantages / weaknesses of content analysis
-
Some documents may not be accessible
-
Documents may not be representative
-
Cannot study unrecorded events
-
Documents may be incomplete
-
Less suitable for making comparisons
-
Susceptible to coder bias
Text analysis
The approach to documentary analysis is referred to as text analysis when the focus is on the text of the document. It sees text as virtual reality and the world as text.
-
Discourse analysis: Precise application of content in a qualitative context. It deals with communication, text, language, talk and conversation. It deals with group discussions, interview transcripts and policy documents.
-
Hermeneutic: Hermeneutic focuses on text interpretation. This includes both grammatical as well as psychological interpretation.
Variables, sampling, hypothesis, reliability and validity
Variables
Variables are empirical constructs that take more than one value.
Variables must relate to one concept only and must be measurable
Types of variables
-
Independent variable: Variable that is set to cause changes in or explain another variable. It is that factor that scientist manipulates. The presumed cause.
-
Dependent variable: Variable that is set to be affected or explained by another variable. It is the factor the scientist observes. The presumed effect.
-
Extraneous variable: These are variables outside the research question, argument or hypothesis
-
Discrete variable: Not continuous and uses whole units only
-
Continuous variable: Continuous and can be fractioned indefinitely.
-
Demographic variables: Deal with demographic data.
Sampling
A sample is a subset of the population that represents the entire group and sampling is the procedure employed to extract samples for sampling
Advantages of sampling
-
Best suited when not possible to study large population.
-
Low on cost
-
Less time consuming
-
Increases accuracy of data
-
Greater response rate
-
Greater co-operation from respondents
Disadvantages of sampling
-
Chances of bias
-
Difficulties in selecting a truly representative sample
Principles of sampling
-
Sample units must be selected in a systematic and objective manner
-
Easily identifiable and clearly defined
-
Independent from each other, uniform and of same size and should appear only once in the population
-
Not interchangeable
-
Once selected cannot be discarded
-
Selection process should be based on sound criteria
-
Researchers should adhere to the principle of research
Types of sampling
-
Probability (random) sampling: Every unit of the target population has an equal, calculable and non-zero probability of being included in the sample.
-
Simple random sampling: Sampling units apart from having an equal chance of being selected are independent of each other.
-
The lottery method
-
The random numbers method
-
The computer method
-
Systematic random sampling: Sampling units are chosen randomly as in simple random sampling but in this the random choice is also integrated with the choice of another sampling unit.
-
Stratified random sampling: Target population is divided into a number of strata and sample is drawn from each stratum
-
Multi-stage sampling: Selection of a large sample with subsequent new samples taken in succession from those previously selected samples.
-
Cluster sampling: Researcher chooses the study units progressively beginning with clusters and then moving on to smaller groups within them before the final sampling unit is considered.
-
Multi-phase sampling: Identical to multi-stage sampling with the difference that in this sampling procedure each sample is adequately studied before the next sample is drawn.
-
Area sampling: Multi-stage sampling is applied in geographical area.
-
Spatial sampling: Employed when the study addresses people temporarily congregated in a space and the data have to be collected before the crowd is dispersed.
-
Non-probability sampling: Do not employ the rule of probability, do not ensure representativeness and the degree to which sample differs population remains unknown.
-
Accidental or convenience or haphazard sampling: Sample units are those people which accidently come into contact with the researcher.
-
Purposive or judgmental sampling: Researchers purposely chose subjects who in their opinion are relevant to the project
-
Quota sampling: Researcher sets a quota of respondents to be chosen from specific population groups defining the basis of choice. The researcher considers all significant dimensions of the population and ensures that each dimension will be represented in the sample.
-
Snowball sampling: Researcher chooses a few respondents, using accidental sampling or any other method and asks them to recommend other people who meet the criteria of the research and who might be willing to participate in the project.
-
Theoretical sampling: Sample units are not chosen by the researcher prior to the commencement of the study but determine by the knowledge that emerges during study.
Sample size: The sample must be as large as necessary and as small as possible.
Determinants of sample size
| Underlying methodology | Nature of the study object | Available time and resources |
|---|---|---|
| Homogeneity of the target population | Accuracy | Nature of the data required |
| Purpose, intensity and nature of the study | Response rate |
Estimating sample size
-
Parten table:
-
0.05 confidence level
-
0.0 confidence level
-
Krejcie and Morgan table:
Based on population size for population ranging from 10 to 1000000
Hypothesis
A hypothesis is an assumption about relations between variables. It is a tentative explanation of the research problem or a guess about the research outcome.
Theodorson and Theodorson - A hypothesis is a tentative statement asserting a relationship between certain facts.
Kerlinger - A hypothesis is a conjectural statement about something, the validity of which is unknown.
Webster - Hypothesis is an assumption made in order to draw out and test its logical and empirical consequences.
Problems in formulating hypothesis – Goode and Hatt
-
Absence of a clear theoretical framework
-
Lack of ability to utilize that theoretical framework logically
-
Failure to be acquainted with available research techniques so as to be able to phrase the hypothesis properly.
Functions / importance of hypothesis – Sarantakos
-
To guide social research by offering direction to the structures and operations.
-
To offer a temporary answer to research question
-
Facilitate satisfied analysis of variables in the context of hypothesis testing.
Other functions / importance
-
Tools of scientific inquiry
-
Tools for the advancement of knowledge
-
To test theories
-
To suggest theories
-
Describe social phenomenon.
Secondary functions / importance
-
Help formulating social policy
-
Assist in refuting certain common sense notions.
-
Indicate need of change in systems and structures by providing new knowledge.
Sources / origin of hypothesis
| Cultural values of society | Past research | Folk wisdom | Discussion and conversations |
|---|---|---|---|
| Personal experiences | Intuition | Science | Analogies |
Criteria for hypothesis construction
-
It should be empirically testable
-
It should be specific and precise
-
Statements in the hypothesis should not be contradictory
-
It should specify variables between which the relationship is to be established
-
It should describe 1 issue only
Nature of hypothesis
-
It must accurately reflect the relevant sociological fact
-
It must not be in contradiction with approved relevant statements of other scientific disciplines
-
It must consider the experience of other researchers.
Types of hypothesis
-
Working hypothesis: Preliminary assumption by the researcher about the research topic particularly when sufficient information is not available to establish hypothesis.
-
Scientific hypothesis: Contains statement based on or derived from sufficient theoretical and empirical data.
-
Research hypothesis: It is a researcher’s proposition about some social fact without reference to its particular attributes.
-
Alternative hypothesis: Set of 2 hypothesis (research and null) which states the opposite of null hypothesis.
-
Null hypothesis: It is reverse of research hypothesis. It is a hypothesis of no relationship. Does not exist in reality but are used to test research hypothesis.
-
Statistical hypothesis: According to Winter it is a statement about statistical population that one seeks to support or refute. Things are reduced to numerical quantity and decisions are made about these quantities.
Reliability and validity
Validity
According to Carmines and Zeller validity is the extent to which an instrument measures what it is supposed to measure.
Types of validity
-
Empirical validity: A measure is taken to have empirical validity if the findings are supported by already existing empirical evidence.
-
Theoretical or conceptual validity: A measure is taken to have theoretical validity if its findings comply with the theoretical principles of the discipline, i.e. if they do not contradict already established rules of the discipline.
-
Face validity: If on the face of it, it measures what it is expected to measure
-
Content validity: A measure is said to have content validity if it covers all possible dimensions of the research topic.
-
Construct validity: A measure can claim construct validity if its theoretical construct is valid
-
Internal validity: It refers to the extent to which the research design impacts on the research outcomes.
-
External validity: It refers to the extent to which the research findings can be generalized and is mostly relevant to explanatory studies.
Reliability
According to Carmines and Zeller, “Reliability” is the extent to which a measurement instrument or procedure yields the same results on repeated trial.
3 types of reliability by Kirk and Miller
-
Quixotic reliability: This refers to the circumstances in which a single method of observation yields the same measurement over and over again. In an ethnographic study, this kind of "reliability" of data indicates that the investigator has managed to elicit "rehearsed" or "politically correct" information.
-
Diachronic reliability: This refers to the stability of an observation over time.
-
Synchronic reliability: This refers to the similarity of observations within the same time period, which can be evaluated by comparisons of the same data by different methods.
| Validity | Reliability |
|---|---|
| Measure of the quality of measurement | Measure of the quality of consistency |
| Measures relevance, precision and accuracy | Measures objectivity, stability, consistency |
| Answers the questions; does the instrument measure what it is supposed to measure | Answers the question; does the instrument produce the same results every time it is employed |
Reliability is a necessary but insufficient condition for validity in research. Reliability is a necessary pre-condition for validity. Validity maybe a sufficient but not necessary condition for reliability. Both complement each other but are not coterminous. |
Quick Revision Map — Unit 3
- 1. Qualitative and quantitative methods
- 2. Techniques of data collection
- 3. Other methods
- 4. Naturalistic research
- 5. Documentary method
- 6. Variables, sampling, hypothesis, reliability and validity
Preparing Sociology Optional? Build Your Foundation with Legacy IAS — Bangalore
Expert faculty, structured GS & Optional guidance, and Bangalore’s most trusted UPSC coaching — all under one roof.


