Purpose
Recently, I was working on a centralized explicit proxy service for one of my customers using a well-known Squid Internet proxy.
The infrastructure is built on top of a shared AWS account hosting all standard infrastructure services, such as Internet Proxy, SOCKS5 Proxy, DNS Resolvers, ADDC, Centralized logging, and much more.
Consumers are using the Internet Proxy service from multiple child AWS accounts and regions within the client AWS Organization thanks to AWS PrivateLink.
Trade-off
When it comes to logging proxy activities from consumers, we encountered a pain point in which the client’s source IP address was the IP addresses of the Network Load Balancer (NLB) instead of the real consumer’s workload IP address.
It was lost in the extended network path:
VPC Endpoint → VPC Endpoint Service → NLB → TargetGroup -> Proxy Instances
This classic behavior leads to unknown usage of our service, making it challenging to troubleshoot unattended usage.
This is especially problematic when it impacts other clients (heavy load, saturation), and you need to manage the Quality of Service (QoS) of this critical Internet access, because at some point its now a Single Point Of Service (SPOF).
Solution
The Proxy Protocol (PP) is a simple way to pass a client’s connection details, like their IP address, through various proxies or NATs to the final server.
It helps ensure the server knows the original client’s information, even if the connection passes through multiple layers. This is especially useful for logging, security, and other features that depend on knowing the true origin of a request.
The protocol is efficient and doesn’t require significant changes to your existing setup, making it easy to implement and maintain.
There is an existing option at the AWS TargetGroup level to activate this PPv2 in conjunction with your NLB.
Using Proxy Protocol V2, you will be able to have the following additional information:
- Client IP address
- Source VPC Endpoint ID
Architecture Schema
Squid Configuration
At the cost of changing only two lines of Squid configuration, you will be able to enable Squid to properly handle this new HTTP header.
The default log format will automatically handle the proper source IP coming from this header.
Requirements:
Squid version ≥5
Squid Configuration:
http_port 3128 require-proxy-header
proxy_protocol_access allow localnet
Conclusion & Key Takeaways
Preserving client IPs when using AWS PrivateLink with proxies is essential for accurate logging, security, and troubleshooting. By enabling Proxy Protocol v2 on your NLB and making minor adjustments to your Squid configuration, you can maintain visibility into the true origin of requests, even in a multi-account architecture. This solution is easy to implement and greatly improves your ability to manage and monitor proxy traffic effectively.
It’s available for Squid, but also for well-known other apps: Apache, Citrix ADC, Exim, Envoy Proxy, HAProxy, and many other.
That’s all, folks! 👋🏼
zoph.