After a lot of investigation into Node.js, I found that the problem of IPv4 address used instead of IPv6 is Node's spec. For reference, here is a explanation of verbatim option of dns.lookup:
verbatim <boolean> When true, the callback receives IPv4 and IPv6 addresses in the order the DNS resolver returned them. When false, IPv4 addresses are placed before IPv6 addresses. Default: currently false (addresses are reordered) but this is expected to change in the not too distant future. New code should use { verbatim: true }.
I suppose Joplin doesn't call dns.lookup, but dns.lookup is called internally in Node when Joplin accesses to servers. In addition, there is no way to set/configure default value of dns.lookup's verbatim option.
The conclusion is that I (and Joplin) almost can do nothing to fix the problem except just waiting for the "not too distant future" of default value of verbatim to be true. (And it might not be very soon...)
PS. In my opinion, that Node's behaviour is (although it has been good when IPv6 was not very common) very outdated and doesn't match with RFC, and should be fixed soon... (Yes, this has nothing to do with Joplin.)