Nginx is an open source, and high-performance web server application that can do load balancing, caching, reverse proxy, and more. Nginx is known to be reliable, fast, efficient, and stable modern web server. It can handle thousands of requests in seconds without consuming much hardware resources in the server. You can use Nginx as:
- Web Server: To server your website static and dynamic content.
- Reverse Proxy: It acts as a gateway between servers and clients. Example is Node.js and Python backend applications proxying.
- Load Balancer: It does the distribution of traffic coming in across multiple backend servers for efficiency and better resource utilization
- Caching: It stores content that’s accessed more frequently to increase response times
Nginx as TCP Load Balancer
Nginx can also be used as a TCP Load Balancer. In this mode it distributes the incoming TCP traffic on configured port across defined backend servers. This is commonly used to load balance traffic for services like mail servers, databases, logging servers, among others.
In this article we shall configure Nginx as Load balancer to our Kubernetes API servers (control plane) listening on port 6443.
Set up Nginx as Kubernetes API Load Balancer
Follow the following steps to set up Nginx as a Load Balancer to your Kubernetes API server.
1 – Install Nginx
Begin with the installation of Nginx web server in your system.
Install Nginx on Ubuntu / Debian
Ensure the necessary dependencies are installed before adding the repository.
sudo apt update
sudo apt install curl gnupg2 ca-certificates lsb-release ubuntu-keyringImport GPG key used in package signing.
curl -fsSL https://nginx.org/keys/nginx_signing.key|sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/nginx.gpgEnable Nginx official APT repository
- On Ubuntu systems
echo "deb http://nginx.org/packages/mainline/ubuntu `lsb_release -cs` nginx"|sudo tee /etc/apt/sources.list.d/nginx.list- On Debian systems
echo "deb http://nginx.org/packages/mainline/debian `lsb_release -cs` nginx"|sudo tee /etc/apt/sources.list.d/nginx.listInstall nginx package using apt command
sudo apt update && sudo apt install nginxInstall Nginx on Enterprise Linux systems – Rocky / AlmaLinux / CentOS Stream
Disable Nginx AppStream repository.
sudo dnf module disable -y nginxAdd official Nginx RPM repository which provide latest packages.
sudo tee /etc/yum.repos.d/nginx.repo<<EOF
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/rhel/\$releasever/\$basearch/
gpgcheck=0
enabled=1
EOFUpdate DNF package index
$ sudo dnf makecache
nginx repo 670 kB/s | 34 kB 00:00
Rocky Linux 9 - BaseOS 20 kB/s | 4.1 kB 00:00
Rocky Linux 9 - AppStream 22 kB/s | 4.5 kB 00:00
Rocky Linux 9 - Extras 14 kB/s | 2.9 kB 00:00
Metadata cache created.Install Nginx from the repository added:
sudo dnf -y install nginxHit y key when asked to confirm installation of nginx package.
Dependencies resolved.
===================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
===================================================================================================================================================================================================================================================================================
Installing:
nginx x86_64 1:1.26.1-2.el8.ngx nginx 962 k
Installing dependencies:
compat-openssl11 x86_64 1:1.1.1k-4.el9_0 appstream 1.5 M
Transaction Summary
===================================================================================================================================================================================================================================================================================
Install 2 Packages
Total size: 2.4 M
Installed size: 6.9 M
Is this ok [y/N]: y2 – Configure Nginx as Load Balancer for Kubernetes API
Open the default nginx configuration file.
sudo vim /etc/nginx/nginx.confAdd the following line after the ending } – “add to very end of file”
include /etc/nginx/tcpconf.d/*.conf;Here is a sample complete configuration file after adding the line.
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
include /etc/nginx/tcpconf.d/*.conf;Create a directory that will store Nginx configuration files for TCP load balancing.
sudo mkdir /etc/nginx/tcpconf.dNow we can create the configuration file specific to Kubernetes Load balancing
sudo vim /etc/nginx/tcpconf.d/kubernetes_tcp.confThese are the contents added to the configuration file; adjust to your ideal environment setup.
stream {
upstream kube_apiservers {
least_conn;
server 10.25.25.11:6443 weight=1 max_fails=3 fail_timeout=10s;
server 10.25.25.12:6443 weight=1 max_fails=3 fail_timeout=10s;
server 10.25.25.13:6443 weight=1 max_fails=3 fail_timeout=10s;
}
server {
listen 6443;
proxy_pass kube_apiservers;
proxy_timeout 10m;
proxy_connect_timeout 1m;
}
}Validate Nginx configuration file to confirm there are no syntax errors:
$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successfulEnable nginx service and restart after making the changes.
sudo systemctl enable nginx && sudo systemctl restart nginxCheck status of the service to ensure it’s running.
systemctl status nginx3 – Allowing the port in firewall
For RHEL based systems with firewalld
sudo firewall-cmd --add-port=6443/tcp --permanent
sudo firewall-cmd --reloadFor Debian based systems with UFW:
sudo ufw allow 6443/tcp
sudo ufw reload4 – Labeling port on SELinux
If you have SELinux in enforcing operation mode, label port 6443 for TCP traffic with Nginx by running the following command:
sudo dnf install policycoreutils-python-utils python3-semanage
sudo semanage port -a -t http_port_t -p tcp 6443Explanations to the options used:
-a: Used to add a new port-t http_port_t: Specify SELinux type for HTTP traffic, which allows NGINX to use this port.-p tcp: Specifies the protocol used
To validate run:
sudo semanage port -l | grep 6443Also allow Nginx http server to make network connections.
sudo setsebool -P httpd_can_network_connect 1Verify the setting, you should see --> on
sudo getsebool httpd_can_network_connect
httpd_can_network_connect --> onRestart nginx service
sudo systemctl restart nginxYou can identify any denied operations by checking audit logs if you encounter issues with SELinux
sudo ausearch -m avc -ts recent5 – Test Kubernetes port
Check if nginx is bound on port 6443
$ sudo ss -tunelp|grep 6443
tcp LISTEN 0 511 0.0.0.0:6443 0.0.0.0:* users:(("nginx",pid=53534,fd=7),("nginx",pid=53533,fd=7),("nginx",pid=53532,fd=7)) ino:351840 sk:1002 cgroup:/system.slice/nginx.service <->If you have telnet client you can use it to validate connection:
$ telnet 127.0.0.1 6443
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.From this point you can proceed to bootstrap your Kubernetes cluster if doing a new installation. Remember to define an A record and map DNS name to the load balancer IP address for easy updating or changing of load balancer at later stage. Avoid binding your Kubernetes control plan endpoint to an IP address.
Offloading Kubernetes administration tasks enables you to focus on growing your business and brand. Allow CloudSpinx Engineers to handle your Kubernetes clusters, and you give attention to the business growth!
