This is completely up to the relying party (i.e. website) and definitely possible. WebAuthN makes explicit accomodations for a user having multiple credentials registered on their device, and they can even be of different types (e.g. you can have one shared hardware authenticator, your iCloud Keychain passkey and your wife's Bitwarden-stored passkey all registered on the same account).

I have multiple passkeys registered with some sites, e.g. Github (and conversely I use many sites that only allow me to associate a single phone number, which I better never lose access to...)

This seems like the standard to allow multiple passkeys, which makes sense... the passkeys are device specific, (not considering synced ones like bitwarden)... So if i login on my windows pc with a passkey i also want to be able to login on my iphone with a passkey, if they didn't support multiple passkeys per account this wouldn't be possible, even without taking into account different people in a family having a passkey.

Passkeys are very much intended to be synced by various WebAuthN stakeholders, for better or worse.

But yes, any site that e.g. wants to support roaming authenticators ("Yubikeys") needs to support multiple credentials, or recovery from a device loss gets very tricky.