The most important rule is never to change the only copy of the source file. Work on a copy and produce a separate output file.
1. Profile before transforming
Raise encoding, separator, number of rows and columns, headers, missing values and observed types. This photograph will verify that cleaning did not delete unexpected data.
2. Write a minimum contract
| Column | Type | Mandatory | Rule |
|---|---|---|---|
| customer id | text | yes | Single, not empty |
| created at | date | yes | ISO 8601 format |
| amount | decimal | No | decimal point, positive value |
3. Normalize with caution
- remove peripheral spaces without altering meaningful content;
- standardize dates to an unambiguous format;
- preserve initial identifier zeros;
- define a unique representation for missing values;
- do not automatically convert a text into a number without a contract.
4. Treat duplicates
Start with strictly identical lines, then apply the business key. Keep a log indicating the rule and number of lines removed. The quasi-doubles must be reviewed separately.
5. Validate output file
Count rows, check rejected values, check mandatory columns and open a sample with the target system. Cleaning is completed only when deviations are explained.
First diagnosis
DataCheck structure, missing values, duplicates, mixed types and risk cells.
Analyze CSVSee also validate a CSV file and detect duplicates.
Published on 19 July 2026.