COLUMN - Updated 2026-06-26
Why Excel Breaks UTF-8 CSV Files
A practical explanation of why UTF-8 CSV files can look broken in Excel, when BOM helps, and how to split Excel copies from upload-safe CSV files.
Open CSV Encoding FixerThe short answer is that CSV does not reliably carry its own encoding label. A file can be valid UTF-8, but Excel may still open it through a different assumption, especially when the file has no UTF-8 BOM or when the operating environment has legacy regional defaults. The result looks like data corruption even when the original bytes are still recoverable.
For AdSense-quality content, the useful answer is not just "add BOM." The workflow should separate the purpose of the file: one copy for long-term storage and automated upload, another copy for Excel review, and a documented path for legacy systems that expect Shift-JIS, CP949, or another non-UTF-8 encoding.
The Hidden Contract in Every CSV
A CSV file is plain text with delimiters. It usually tells you where rows and columns are, but not necessarily how bytes should become characters. That hidden contract is negotiated by the program that opens the file.
| Reader | Common assumption | What can go wrong |
|---|---|---|
| Browser or API | UTF-8 is common | BOM may be treated as part of the first field by weak importers |
| Excel double-click | Regional and file hints matter | UTF-8 without BOM may be guessed as a legacy encoding |
| Legacy Japanese system | Shift-JIS or CP932 may be expected | UTF-8 text can be rejected or shown as mojibake |
| Legacy Korean system | CP949 or EUC-KR may be expected | UTF-8 Korean text can be garbled |
| Data warehouse import | Usually configurable | delimiter, quote, and BOM rules must be explicit |
That is why the same CSV can look correct in a code editor, broken in Excel, and rejected by an upload form.
BOM Is a Compatibility Signal, Not a Universal Fix
The UTF-8 BOM is useful when the immediate user is an Excel user who expects to double-click a CSV and read it. It gives Excel a strong hint that the file should be interpreted as UTF-8.
The same BOM can be harmful in a different destination. Some importers treat the BOM as an invisible character before the first column name. Then email becomes something closer to \uFEFFemail, and downstream joins, schema mapping, or validation rules fail.
| Destination | Safer copy | Reason |
|---|---|---|
| Excel review by non-developers | UTF-8 BOM CSV | Better chance of readable first open |
| API upload | UTF-8 without BOM unless docs say otherwise | Avoids hidden first-field characters |
| Database import | Follow importer config | Encoding and delimiter should be explicit |
| Japanese legacy exchange | Shift-JIS copy if required | Destination may not accept UTF-8 |
| Archive copy | Original + normalized UTF-8 | Keeps recovery path open |
The practical rule is simple: never let an Excel compatibility copy become the only source of truth.
A Safer Team Workflow
- Save the original file unchanged.
- Inspect a small sample with CSV Encoding Fixer.
- Check visible text, column count, delimiter, quotes, and line endings together.
- Create a separate Excel copy only when people need to open the file in Excel.
- Create a separate upload copy that follows the receiving system's documented rules.
- Put the encoding choice in the filename or handoff note.
Example filenames:
orders_original_2026-06-26.csv
orders_excel_utf8-bom_2026-06-26.csv
orders_upload_utf8_2026-06-26.csv
orders_legacy_shift-jis_2026-06-26.csv
This looks fussy until the first time a team spends an afternoon asking whether a broken value was corrupted by Excel, by an upload form, or by the export job.
How to Diagnose the Failure
| Symptom | Likely cause | First check |
|---|---|---|
| Japanese text becomes mojibake | Shift-JIS and UTF-8 mismatch | Try Shift-JIS and UTF-8 previews |
Korean text becomes ì or ê patterns | UTF-8 read as legacy encoding | Try UTF-8 and CP949/EUC-KR previews |
| All values appear in one column | delimiter mismatch | compare comma, tab, semicolon |
| First header fails mapping | BOM seen as data | inspect first column name |
| Text is readable but rows are shifted | quotes or embedded delimiters | test fields with commas and line breaks |
Encoding is only one part of CSV safety. Delimiters and quoting are just as important because a file can be readable and still structurally wrong.
FixData Workflow
Use CSV Encoding Fixer before opening the production file in Excel:
- Paste a sanitized sample with headers and 10-30 rows.
- Compare the preview against known names, addresses, and notes.
- Check the detected row and column shape.
- Download a UTF-8 BOM copy for Excel review if needed.
- Keep the original and upload copy separate.
The tool should not become a reason to paste private production data into random documents. Use a small sanitized sample when the goal is diagnosis.
Practical FAQ
Should every UTF-8 CSV include BOM?
No. BOM is useful for Excel compatibility in many workflows, but it is not universally required. Use it for Excel copies, and follow destination documentation for APIs and imports.
Is mojibake always permanent corruption?
Not always. If the original bytes are intact, reopening with the right encoding can recover the text. If someone saved the already-garbled text over the original, recovery becomes much harder.
Why does Excel sometimes work after using import instead of double-click?
The import flow lets you choose the encoding and delimiter instead of relying on a guess. That is why it is safer for multilingual CSV files.
What should a support ticket include?
Include the source system, destination system, OS, Excel version if relevant, expected text, observed text, delimiter, and a sanitized sample. Do not include customer data or private exports.
Related Reading
- Fix Japanese CSV mojibake
- Shift-JIS to UTF-8 CSV conversion
- CSV and Excel encoding center
- Excel character encoding for CSV
Updated
2026-06-26
Official references
These official references were used to verify the criteria discussed in this article.
- Import or export text (.txt or .csv) filesMicrosoft Support - Checked: 2026-06-26
- Encoding StandardWHATWG - Checked: 2026-06-26
- TextDecoder - Web APIsMDN Web Docs - Checked: 2026-06-26
- The Unicode StandardUnicode Consortium - Checked: 2026-06-26