Secondary Data Research Limitations: 12 Common Mistakes When Using Published Datasets for Research
By Siddharth Gupta, independent data consultant | Last reviewed: 31 October 2026
Secondary data research limitations can undermine an otherwise careful dissertation if the dataset is not checked before analysis. A dataset can be published, cited and free, and still be wrong for your question. Below are the 12 mistakes I see most often, a 7-stage audit to run before modelling, and a seven-stage audit checklist below you can keep.
Quick Answer: The 12 Most Common Secondary Data Research Limitations
- Trusting variable names without reading definitions
- Skipping the codebook and technical report
- Ignoring who was sampled and why
- Reading missing codes as real values
- Assuming a measure never changed
- Mixing releases without a version log
- Ignoring revisions and data vintages
- Missing reclassifications
- Pooling waves without a comparability test
- Ignoring survey weights and design
- Merging at mismatched units
- Writing a one-line limitations paragraph
To catch these secondary data research limitations early, run the 7-stage audit below, score each stage red, amber or green, and decide before you model.
Why Secondary Data Research Limitations Matter Before You Open R or SPSS
Secondary data research limitations are the constraints that come from analysing data collected by someone else for a different purpose, such as unclear variables, thin documentation and changes in measurement over time.
Published does not mean analysis-ready, because the original team collected that data to answer their question, not yours. In my experience, examiners notice unaddressed secondary data research limitations, and they are hard to defend at a viva.
If you are still choosing between your own data and existing data, start with my guide on the advantages and limitations of secondary data.
Short on time? Start with Stages 2 and 3, because documentation and variable definitions cause most of the damage I see. A popular dataset is not automatically a safe one, since many papers reuse the same file with the same unchecked assumption. This applies at master’s level too, because in my experience examiners ask the same questions.
Dataset Documentation Problems in Research (Mistakes 1 to 4)
Most dataset documentation problems in research begin with one habit: students open the CSV before they open the documentation.
Mistake 1: Trusting Variable Names Without Reading Definitions
A column called “income” may mean personal or household income, before or after tax, weekly or yearly. Read the exact survey question, the unit and the response options, because I never accept a variable name as a definition. If the variable does not match your construct, see my note on checking whether dataset variables answer the research question.
Mistake 2: Skipping the Codebook, README and Technical Report
Missing metadata is a quality signal in itself. The American Political Science Review expects authors to supply a codebook for each dataset, or a README pointer to public documentation, and to include secondary datasets in raw form with the code that transformed them. Read the journal’s reproducibility policy as a model of what examiners quietly expect.
If I cannot find a codebook, questionnaire and technical report in ten minutes, I treat the dataset as a red flag.
Mistake 3: Ignoring Why the Data Were Collected and Who Was Sampled
Check the population, sampling frame and response rate, because a health survey of adults over 50 cannot speak for all adults. These secondary data research limitations decide what you are allowed to claim.
Mistake 4: Treating Special Missing Codes as Real Values
Codes such as -9, 99 or 999999 often mean “refused” or “not applicable”, and software will happily average them with real values. Recode them to missing before you describe anything. If your sample sizes then differ across tables, read my note on investigating missing dataset observations.
* Stata
mvdecode income, mv(-9 99 999999)
# R
df$income[df$income %in% c(-9, 99, 999999)] <- NACommon Problems With Secondary Datasets When Time and Versions Change (Mistakes 5 to 8)
These secondary data research limitations stay hidden because the file looks tidy, and the trouble is in the history of the data, not in the rows.
Mistake 5: Assuming a Measure Meant the Same Thing in Every Year
Census 2000 let respondents choose more than one race for the first time, and the Census Bureau noted that this made direct comparison with the 1990 race question difficult. Read the Census Bureau’s release and always check change notes before comparing years.
Mistake 6: Mixing Data Releases Without a Version Log
Two downloads of the same dataset can differ, so note the version, release date and file name. Keep the raw file untouched and work on a copy.
Mistake 7: Ignoring Revisions and Data Vintages
The ONS publishes a first estimate of quarterly GDP six weeks after the quarter ends, and its data production cycle takes three years to complete. See the ONS note on timeliness and accuracy. The ONS later revised up its estimate of 2022 GDP growth in the 2023 Blue Book, after annual business survey data showed weaker growth in firms’ intermediate inputs than the early estimates assumed. Record the vintage you used, or your “finding” may only be a revision.
Mistake 8: Missing Reclassifications
Disease codes, industry codes and boundaries get redefined. When the US moved from ICD-9 to ICD-10 for deaths, influenza and pneumonia had a comparability ratio of about 0.70, so that cause was roughly 30% less likely to be chosen as the underlying cause. The CDC’s comparability note shows why a crosswalk is not optional.
If your question is about change over time, choose the design deliberately, as I explain in cross-sectional vs longitudinal design.
Secondary Data Research Limitations in Pooling, Weighting and Merging (Mistakes 9 to 11)
This is where I see the most expensive errors, because software gives no warning.
Mistake 9: Pooling Waves or Sources Without Testing Comparability
Pooling is not just stacking rows. CLOSER’s metadata guide says user guides and original questionnaires explain, for example, why a variable has many missing values because of survey logic, and how interview mode can change the way people answer. Read the CLOSER metadata guide, then compare distributions by wave and add wave indicators. If the waves use different coding systems, start with my guide to checking coding and data transformation errors.
Mistake 10: Ignoring Survey Weights, Clustering and Stratification
Unweighted estimates can misrepresent the population, and standard errors may be too small. Read how the sample was drawn, because design is where secondary data research limitations bite hardest. My note on stratified vs cluster sampling explains why design matters.
Mistake 11: Merging at Mismatched Units
Joining district-level data to individual-level data is a common trap, because you then describe individuals using an area average, which is the ecological fallacy. Confirm the unit of analysis and merge key, and check the structure of your tabular data before joining.
What Published Papers Say, and Where I Disagree
Both papers below deal with secondary data research limitations, and my comments are based on the abstracts available to me.
Fleischer and Khalil, writing in Children’s Health Care in 2023, examine limits in electronic medical records and national survey data, including the US Census, and how demographic data are collected and interpreted. See the paper’s record at Drexel. I agree that demographic categories are a serious weak point, but advocacy-level recommendations do not help a student with a dataset open tonight.
Hari Luitel’s 2018 conference paper suggests judging secondary data by its original purpose, its methods, the author’s credentials and cross-checking of numbers, and it treats primary data as generally more accurate. Here is the conference abstract. I agree with purpose and cross-checking, but I disagree on credentials and on primary data being more accurate, because a well-documented national survey often beats a small convenience sample.
Limitations of Publicly Available Datasets You Must Admit in Your Dissertation
Mistake 12: Writing a One-Line, Generic Limitations Paragraph
Most dissertations understate secondary data research limitations in a single line, such as “secondary data may not be accurate”, which tells an examiner nothing. The limitations of publicly available datasets must be specific: which variable, which bias, and what you did about it.
What Examiners Look For
In my experience, examiners reward honesty about secondary data research limitations when it is precise. A named limit with a stated response looks like control.
A Limitations Paragraph You Can Adapt
How to Assess Public Datasets for Research: A 7-Stage Dataset Quality Audit
A University of Pittsburgh library guide groups dataset quality into availability, usability, reliability, relevance and presentation quality. The Pitt library guidance is a good base, and I turned those ideas into seven steps that catch secondary data research limitations before modelling:
- Provenance, licence and permissions
- Documentation and metadata
- Variables and definitions
- Time, version and revision
- Structure, design and weights
- Integrity checks
- Fitness-for-purpose decision and audit log
Start With Datasets That Document Themselves
The UK Data Service lets you read a study’s documentation in its catalogue before you decide to use the data. ICPSR advises judging fit through the study homepage and accompanying documents such as the codebook or questionnaire. In the UK, the UK Data Service holds studies such as Understanding Society, and in the USA, ICPSR, the CDC’s NCHS and NCES are good starting points.
Standards help too, because the DDI standard describes surveys, questionnaires and data files in a consistent way, and FAIR principles explain why good documentation makes data reusable.
Stage 1: Provenance, Licence and Permissions
Who produced the data, when, and under what licence? Confirm how to cite it and if your ethics committee must approve its use, and remember that UK researchers must also think about UK GDPR if a file holds personal data.
Stage 2: Documentation and Metadata Check
Find the codebook, README, questionnaire and technical report, and confirm that the file you hold matches the documented version.
Stage 3: Variable and Definition Audit
List every variable you will use with its exact definition, unit and valid range, and check that each one measures your construct, not a loose proxy.
Stage 4: Time, Version and Revision Audit
Record the version and vintage, and search for definition changes, breaks and reclassifications across the years you need.
Stage 5: Structure, Design and Weights Audit
Identify the unit of analysis, sampling design and weights, and confirm that any pooling or merging is justified. If the structure cannot support your plan, read about choosing an analysis suitable for the available variables.
Stage 6: Integrity Checks
Check missing codes, duplicates, impossible values and outliers, and compare your counts with the published totals. For unusual values, see identifying unusual observations in datasets, and for odd outputs after importing data, see auditing suspicious statistical results.
* Stata
isid id wave
duplicates report id wave
# R
sum(duplicated(df[, c("id", "wave")]))Stage 7: Fitness-for-Purpose Decision and Audit Log
Decide: proceed, proceed with caveats, or change the dataset. Keep the raw file and your cleaning code, because this log becomes the paragraph that explains your secondary data research limitations.
Secondary Data Quality Assessment: Scoring Your Dataset Before Modelling
Dataset readiness checklist
A simple secondary data quality assessment needs only three colours, and the score also shows how serious your secondary data research limitations are.
| Stage | Green | Amber | Red |
|---|---|---|---|
| Documentation | Full codebook and report | Partial notes | None found |
| Variables | Definitions match construct | Close proxy | Unclear meaning |
| Time and versions | One version, no breaks | Breaks documented | Unknown changes |
| Structure | Weights and units clear | Minor gaps | Unclear design |
| Integrity | Clean after recoding | Fixable issues | Unexplained errors |
Any red in documentation or variables means fix it or change the dataset. Amber is acceptable if you report it, and green everywhere is rare, so do not wait for it.
Take this secondary data research limitations audit with you: use the dataset audit checklist below. If the audit exposes problems you cannot solve, read about the signs you need a dissertation expert.
FAQ
What are the limitations of secondary data?
The data were collected for another purpose. This creates secondary data research limitations such as unclear variables, missing metadata, changed measures and unknown sampling choices, and you cannot ask respondents a new question.
Is secondary data reliable enough for a dissertation?
Yes, if you audit it first. Reliability depends on how you handle secondary data research limitations, especially documentation, version control and fit with your question, and a well-audited public dataset often beats a small primary sample.
What is a codebook and why does it matter?
A codebook lists each variable with its definition, coding and valid values. Without it, you are guessing what the numbers mean, so it is the first file I look for.
Can you combine datasets from different years?
Yes, but only after testing comparability. Check definitions, sampling, survey mode and weights, and include wave indicators; if measures changed, analyse the periods separately.
How do you handle missing data in a secondary dataset?
First, find how missing values are coded and recode them properly. Then check how much is missing and whether it looks random, and report your approach in the limitations section.
How do I write secondary data limitations in my dissertation?
Be specific. Treat secondary data research limitations as findings about your data, not as confessions, and name the variable, the possible bias and your response using the template above.
What are the sources of secondary data?
Common sources are government statistics, university data archives and published studies. Good starting points are the UK Data Service, ICPSR, the ONS, the US Census Bureau, CDC/NCHS and NCES, and you should choose sources that publish a codebook.
Final Thought
Every dataset has limits, and examiners do not mind that, but they do mind when you did not notice them. Audit first, model second, and your dissertation will be easier to defend. That is how I handle secondary data research limitations in every project.
Written by Siddharth Gupta, independent data consultant with 12 years of experience in dissertation and research guidance. I work with R, Stata and SPSS. Connect with me on LinkedIn.