COLUMN - Updated 2026-06-26
Shift-JIS to UTF-8 CSV Workflow for Teams
A team workflow for converting Japanese Shift-JIS CSV files to UTF-8 without losing the original file, breaking Excel review, or creating upload errors.
Open CSV Encoding FixerShift-JIS to UTF-8 conversion is not just a button click. A team has to preserve the original file, identify the real source encoding, check characters that often break, decide whether Excel needs a BOM copy, and confirm the destination system's import rules. Without that workflow, the same file may be converted three times by three people and still fail in production.
This column turns the conversion into a repeatable operating procedure for teams that receive Japanese CSV files from partners, legacy exports, accounting systems, or regional back-office tools.
When to Suspect Shift-JIS
Shift-JIS or CP932 is likely when a file comes from an older Japanese Windows workflow, a legacy business system, or a partner who says the CSV "opens fine in Japanese Excel" but breaks in your web app.
| Signal | What it suggests |
|---|---|
| Japanese names look broken only in the web app | source may be Shift-JIS but app expects UTF-8 |
| File opens in Japanese Excel but fails in API upload | Excel guessed legacy encoding successfully |
| Yen signs, wave dashes, or company names look odd | legacy mapping edge cases may be involved |
| Partner cannot provide export settings | treat encoding as unknown until previewed |
| Same CSV is shared by email and manually edited | protect the original before conversion |
The key is to diagnose from the original file, not from a copy that has already been opened and saved by Excel.
Conversion Decision Table
| Destination | Recommended output | Check before delivery |
|---|---|---|
| Web app upload | UTF-8 without BOM unless docs require BOM | first header, row count, delimiter |
| Excel review | UTF-8 BOM CSV | Japanese preview, column split |
| Legacy Japanese import | Keep Shift-JIS or create Shift-JIS copy | destination spec and unsupported characters |
| Data warehouse | UTF-8 with explicit import config | quote, delimiter, line ending |
| Archive | original + UTF-8 normalized copy | file names and conversion note |
There is no single output that is best for all destinations. A workflow is better than a universal rule.
Step-by-Step Workflow
- Store the original as read-only.
- Make a sanitized sample with headers and representative Japanese rows.
- Preview the sample with CSV Encoding Fixer.
- Compare Shift-JIS and UTF-8 interpretations against known words.
- Convert only a copy.
- Validate row count, column count, delimiter, quotes, and line endings.
- Create a separate Excel copy if reviewers need double-click compatibility.
- Record the source, conversion choice, and destination in a handoff note.
Example handoff note:
Source: partner_export_2026-06-26.csv
Likely source encoding: Shift-JIS
Output for upload: UTF-8, no BOM
Output for Excel review: UTF-8 BOM
Validation: 2,413 rows, 18 columns, comma delimiter, quoted addresses preserved
Known sensitive fields removed from shared sample
Characters Worth Checking
Do not validate conversion with only ASCII headers. Use rows that include real Japanese text patterns.
| Character or value type | Why it matters |
|---|---|
Company suffixes such as 株式会社 | easy visible check for Japanese text |
| Full-width punctuation | often reveals wrong decoding |
| Yen sign and backslash-like values | mapping differences can surprise teams |
| Postal addresses | commas and spaces can affect parsing |
| Product names with symbols | may expose unsupported character paths |
| Empty trailing fields | can reveal delimiter or quote loss |
If the sample only contains id,name,status, you have not tested a Japanese CSV conversion.
Common Failure Modes
| Failure | Cause | Prevention |
|---|---|---|
| Converted file looks correct but upload fails | BOM or header mismatch | inspect first field and destination docs |
| Excel reviewer says the fixed file broke | no BOM copy for Excel | provide a separate Excel review copy |
| Japanese looks correct but rows shift | delimiter or quote issue | validate row and column counts |
| Some symbols change | legacy mapping edge case | check known risky characters |
| The team cannot reproduce the fix | no conversion log | record source, output, and validation checks |
FixData Workflow
CSV Encoding Fixer is useful at the diagnosis and review points:
- before conversion, to compare likely encodings;
- after conversion, to confirm the output is readable;
- before handoff, to create an Excel-friendly BOM copy when needed.
It should not replace destination documentation. If the receiving service requires a specific encoding, delimiter, or BOM rule, that rule wins.
Practical FAQ
Should we convert every Shift-JIS CSV to UTF-8?
For storage and web upload, usually yes. For a legacy Japanese destination, not always. Keep the original and create destination-specific copies.
Can Excel be the conversion tool?
It can be part of review, but it is risky as the only conversion tool because saving can change encoding, delimiters, formatting, and leading zeroes. Always keep the original.
Why did UTF-8 conversion fix text but break the first column?
The output may include a BOM, and the destination importer may treat it as data. Use a no-BOM UTF-8 upload copy when the destination expects it.
What is the minimum QA before sending the file?
Check row count, column count, known Japanese values, delimiter, quote behavior, first header, and destination-specific BOM rules.
Related Reading
- Shift-JIS to UTF-8 CSV conversion
- Fix Japanese CSV mojibake
- CSV and Excel encoding center
- Excel character encoding for CSV
Updated
2026-06-26
Official references
These official references were used to verify the criteria discussed in this article.
- Import or export text (.txt or .csv) filesMicrosoft Support - Checked: 2026-06-26
- Encoding StandardWHATWG - Checked: 2026-06-26
- TextDecoder - Web APIsMDN Web Docs - Checked: 2026-06-26
- The Unicode StandardUnicode Consortium - Checked: 2026-06-26