real_ip_from, or the client address is whatever they say

Behind a proxy, $remote_addr is the proxy — and reading X-Forwarded-For instead is correct only if you also refuse to believe it from anyone but the proxy.

set_real_ip_from 10.0.0.0/8;
set_real_ip_from 172.16.0.0/12;
real_ip_header X-Forwarded-For;
real_ip_recursive on;

# without set_real_ip_from, the client chooses what you log —
# and what your rate limiting and fail2ban rules act on.

This matters for anything that acts on the client address: rate limits, log-reading ban rules, and the IP recorded next to a login. real_ip_recursive walks the chain from the right, skipping trusted addresses, which is what gets the real client when there are two proxies — a CDN in front of a load balancer being the usual case. Getting it wrong means an attacker chooses what you record.