Non-ASCII file names

The problem#

On the Switch’s SD card, file names are parsed as Shift-JIS, below libnx. Hardware run 20 tested 193 names and found a rule that predicts every result:

• A name can be created only when its UTF-8 bytes also form well-formed Shift-JIS. セーブ.txt fails: the last byte of ブ, 0x96, is a Shift-JIS lead byte followed by .. café.txt and ブ.txt happen to work.
• Directory listings leave out every non-ASCII entry, even ones that were created. Those files still open by their exact name.

The runtime and sphaira’s FTP server behave the same way, so the parsing happens below both: [Likely] in Horizon’s FS service. Names written by a PC through a card reader were not tested.

So a game whose files have non-ASCII names cannot be installed reliably, and Ren’Py would not see those files when it scans game/.

What the importer does#

Every exported file whose path has non-ASCII characters is packed into game/rsl-nonascii.rpa, a standard Ren’Py archive, with names in Unicode NFC. Ren’Py reads them from there by their original names. Past 2 GiB, a new archive is started: rsl-nonascii-2.rpa, and so on.

In run 21 a test game had images/セーブ.png, images/背景/ブ.png and audio/ブ.ogg. All three went into the archive, Ren’Py found them (1 OK, 2 OK, 3 OK; archives ['rsl-nonascii']), the images showed and the sound played.

What cannot be packed#

Ren’Py does not load these from an archive, so they stay loose and are still affected. The report warns about each one:

• scripts: .rpy, .rpym and _ren.py;
• .rpa archives.

If a game’s scripts or archives have non-ASCII names, it may not work on the Switch.