pypirc: stipulate UTF-8 encoding

[woodruffw]

This is a kind of interesting/nuanced thing, and I don't know if this PR is actually the right approach 😅 -- I'm opening it for discussion!

Context: .pypirc was never PEP'd; it was seemingly grandfathered in as a distutils implementation detail that gets respected by twine and flit (and perhaps others?). As a result, there's no stipulation as to what charset a .pypirc should be encoded with.

In practice this is mostly a non-issue: most users will default to UTF-8, or in the worst case they'll use whatever system character encoding Python picks and hopefully their consuming tool will too. However this does unfortunately break down sometimes, like with pypa/twine#1268 recently.

I think the best fix here is just to rip the bandaid off and stipulate that .pypirc files should be UTF-8, especially since upcoming Python versions will use UTF-8 in a blanket fashion by default. I've gone with "should" instead of "must" since twine does make an effort to parse using the system charset, but this is a fallback we'd probably like to eventually remove. I'm not sure what flit does, but "should" also felt appropriate given that it's entirely open-ended at the moment 🙂

Open questions:

• Can we just do this, without any additional standardization procedure? I think the answer is yes, since .pypirc is (1) kind of a legacy component anyways, and (2) never went through a packaging PEP (AFAICT). However, I'd like to hear if others feel otherwise.
• Are there other publishing tools that use this file that this needs to be coordinated with? I know twine and flit for certain, and that uv doesn't (yet?) support it. Ref: Feature Request - uv publish recognizes .pypirc file astral-sh/uv#7676

---

📚 Documentation preview 📚: https://python-packaging-user-guide--1980.org.readthedocs.build/en/1980/

[woodruffw]

@pypa/packaging-user-guide-editors does anybody have any objections to this?

I raised it on DPO here as well: https://discuss.python.org/t/psa-proposed-change-to-the-pypirc-spec/105581

(Beyond that, I'm not sure if I need to do anything else procedurally before this can be merged, given that this "spec" is in a legacy/grandfathered category.)

[merwok]

Wording is ok for me, but 1) I would not have the note before the paragraph below (that explains what even this file is) 2) I don’t think a callout («important» markup) is needed.

[woodruffw]

Thanks, I'll adjust!

[ncoghlan]

I think it's fine to just make the change (with the editorial tweaks @merwok suggested).

We have a fair bit of leeway when it comes to things where we're updating the spec to describe reality, precisely because there's no formal certification even of our actual specifications, let alone cases like this where we're operating almost entirely in descriptive mode.