Duplicate checking starts with strictly identical rows. A business-level review should then compare a key, a set of columns or normalized values to identify less obvious repetitions.
Three types of duplicates
| Type | Example | Decision |
|---|---|---|
| Identical row | Every cell has the same value | Often removable after review |
| Identical key | The same order number | Keep the authorized version |
| Near duplicate | A name or address written differently | Apply a business rule or a dedicated matching process |
A safe deduplication method
- Always keep an untouched copy of the source file.
- Identify the business key before deleting anything.
- Normalize spaces, capitalization, dates and numbers when necessary.
- Define which row to keep: the newest, the most complete or the validated one.
- Run the checks again after every transformation.
Spot repeated rows
DataCheck reports strictly duplicated rows as well as structural and data-type anomalies.
Analyze my CSVDo not delete rows automatically
Identical rows can represent separate real-world events. The tool highlights them, but only knowledge of the underlying process can determine whether they are truly unwanted.
For a complete file check, use the CSV validator. To prepare the file before import, follow the guide to clean a CSV file. Before deleting anything, follow the complete method to find CSV duplicates without removing legitimate records.
Frequently asked questions
How do I find duplicates in a CSV?
Start with strictly identical rows, then compare a business key or several normalized columns.
Does DataCheck delete duplicates?
No. It creates a static report and never modifies the uploaded file.
What is a near duplicate?
It is a similar but non-identical value, such as a name or address written differently.
Updated August 22, 2026.