背景: 公司内网繁多, 每个部门的权限不一样, 可以查看的内网等级也不一样. 因此需要在vpn上做权限分离, 实现租户隔离. 方法是采用iptables的SNAT来实现权限控制. 具体如下:

$表示bash shell, #表示注释, > 表示数据库

安装前准备

1、配置阿里云 YUM 镜像

$ wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
$ wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
$ yum clean all
$ yum makecache

2、配置时间同步

$ yum -y install ntpdate
$ ntpdate ntp.aliyun.com

3、关闭 SELinux

$ vim /etc/sysconfig/selinux
SELINUX=disabled

4、 开启内核转发

$ echo net.ipv4.ip_forward = 1 >> /etc/sysctl.conf
$ systcl -p ## 生效

安装 OpenVPN

$ yum -y install openssh-server lzo openssl openssl-devel openvpn \
NetworkManager-openvpn openvpn-auth-ldap zip unzip  easy-rsa iptables-services

创建证书变量

修改下面字段命令，然后保存退出

$ cd /etc/openvpn
$ ln -sv /usr/share/easy-rsa /etc/openvpn/easy-rsa
$ cd /etc/openvpn/easy-rsa/3/
$ vim vars
set_var EASYRSA_REQ_COUNTRY     "CN" ##国家
set_var EASYRSA_REQ_PROVINCE    "BeiJing" ##省份
set_var EASYRSA_REQ_CITY        "BeiJing" ##城市
set_var EASYRSA_REQ_ORG "opsbj" ##组织名称自定义
set_var EASYRSA_REQ_EMAIL       "510749025@qq.com" ##邮箱
set_var EASYRSA_REQ_OU          "Dynamic Times" 

创建证书

建议生产环境的增加ca证书的密码.

$ cd /etc/openvpn/easy-rsa/3/
## 初始化
$ ./easyrsa init-pki

## 创建根证书, 建议生产环境带密码
$ ./easyrsa build-ca nopass

## 签发服务端证书
$ ./easyrsa build-server-full server nopass

# 创建 Diffie-Hellman，确保 key 穿越不安全网络的命令
$ ./easyrsa gen-dh  ##时间稍长,稍等一会
$ openvpn --genkey --secret ta.key

创建服务器配置文件

如下server.conf, 使用的是tap, 可以自由定义客户端ip地址.根据IP地址来进行权限分离.

$ cat server.conf
port 11194
proto tcp-server
dev tap
ca /etc/openvpn/keys/ca.crt
cert /etc/openvpn/keys/server.crt
key /etc/openvpn/keys/server.key  # This file should be kept secret
dh /etc/openvpn/keys/dh.pem
mode server
tls-server
duplicate-cn
keepalive 15 120
comp-lzo
max-clients 150
user openvpn
group openvpn
persist-key
persist-tun
log         openvpn.log
log-append  openvpn.log
verb 5
client-config-dir /etc/openvpn/junhsue/users
server 172.30.0.0 255.255.0.0
#push "dhcp-option DNS 202.96.209.133"
#push "dhcp-option DNS 210.22.84.3"
#push "dhcp-option DNS 114.114.114.114"
## 抽取公共的权限push路由功能, 所有具有vpn的人都可以访问如下内网.
push "route ip 255.255.255.255 " # https://athena.fenghong.tech
push "route ip 255.255.255.255 " # classic jump
push "route ip 255.255.255.255 " # vpc jump
push "route ip 255.255.255.255 " # vpc java testing
push "route ip 255.255.255.255 " # vpc k8s testing
push "route ip 255.255.255.255 " # test db
push "route ip 255.255.255.255 " # test drds
push "route ip 255.255.255.255 " # https://code.fenghong.tech

说明

server 172.30.0.0 255.255.0.0 命令解释：这条命令的结果相当于一系列命令的集合，server用在路由模式; 如果是tap则相当于：

ifconfig 1172.30.0.1 255.255.0.0 设置服务端的tap地址为172.30.0.1

ifconfig-pool 172.30.0.2 172.30.255.254 255.255.0.0 客户端使用的地址池

push "route-gateway 172.30.0.1"

启动服务

$ systemctl start openvpn@server

启动错误, 明明配置文件正确, 修改重载了一下,却发现启动不了.

Job for openvpn@server.service failed because the control process exited with error code.
Failed to start OpenVPN Robust And Highly Flexible Tunneling Application On serve

恢复服务, 需要修复文件. 这个比较蛋疼.

$ fixfiles -R openvpn restore

主机配置SNAT进行权限分离

$ tree                                                                        
.
├── bi
├── data
├── platform
├── product
└── users
    └── louis

有四个组, 比如bi, data,platform,product, 每个组的内网权限不一样.

$ cat bi
push "route 192.168.10.0 255.255.255.0 " # bi-e subnet

$  cat data                                                                    
push "route ip01 255.255.255.255" # data-vpc for data
push "route ip02 255.255.255.255" # offline-testing for data
push "route ip03 255.255.255.255" # risk-vpc-offline for data
push "route ip04 255.255.255.255" # risk-vpc for data
push "route ip05 255.255.255.255" # graphsql for data

$  cat product                                                                                      
push "route ip06  255.255.255.255" # https://hrhx-ymer-admin.fenghong.tech 
push "route ip07  255.255.255.255" # https://debt-admin.fenghong.tech:8443
push "route ip08 255.255.255.255 " # https://bi-report.fenghong.tech:4443
push "route ip09 255.255.255.255 " # https://marketing.fenghong.tech
...

$ cat users/louis  
ifconfig-push 172.30.100.104 255.255.0.0
## >>bi<<
push "route 192.168.10.0 255.255.255.0 " # bi-e subnet

根据group的路由政策. 来进行原目标的SNAT, 进行了权限控制. 去掉了很多, 建议自行根据权限把控来添加MASQUERADE

# Generated by iptables-save v1.6.0 on Sat May 26 10:12:50 2018
*filter
:INPUT ACCEPT [4884159:1326612100]
:FORWARD ACCEPT [3155300:2809181555]
:OUTPUT ACCEPT [4731682:3822395521]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
:DOCKER-USER - [0:0]
-A INPUT -s 10.47.136.50/32 -j ACCEPT
-A INPUT -s 172.16.0.0/16 -j ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
# Completed on Sat May 26 10:12:50 2018
# Generated by iptables-save v1.6.0 on Sat May 26 10:12:50 2018
*nat
:PREROUTING ACCEPT [2:128]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [1:52]
:POSTROUTING ACCEPT [1:52]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.30.10.0/24 -d 192.168.10.0/24 -p tcp -m comment --comment "vpc subnet for bi-e" -j MASQUERADE
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -d 172.16.30.6/32 -o tun0 -j MASQUERADE
-A POSTROUTING -d 192.168.5.0/24 -o tun0 -j MASQUERADE
-A POSTROUTING -d 192.168.11.221/32 -o tun0 -j MASQUERADE
-A POSTROUTING -s 172.30.0.0/16 -d 60.191.0.82/32 -m comment --comment "ip.cn for common" -j MASQUERADE
-A POSTROUTING -s 172.30.1.0/24 -d 47.94.79.97/32 -p tcp -m tcp --dport 80 -m comment --comment "http://cashbus-teldata02.fenghong.tech for spider" -j MASQUERADE
-A POSTROUTING -s 172.30.1.0/24 -d 182.92.140.13/32 -p tcp -m tcp --dport 443 -m comment --comment "https://proxy-service.fenghong.tech for spider" -j MASQUERADE
-A POSTROUTING -s 172.30.2.0/24 -d 192.168.253.39/32 -m comment --comment "vpc pgsql for product" -j MASQUERADE
-A POSTROUTING -s 172.30.22.0/24 -m comment --comment "super administrator" -j MASQUERADE
-A POSTROUTING -s 172.30.0.0/16 -d 60.205.232.231/32  -m comment --comment "vpc jump" -j MASQUERADE
-A POSTROUTING -s 172.30.0.0/16 -d 47.94.188.150/32 -m comment --comment "test db for all" -j MASQUERADE
-A POSTROUTING -s 172.30.255.0/24 -d 123.56.215.244/32 -p tcp -m tcp --dport 443 -m comment --comment "https://wb.fenghong.tech for thirdparty" -j MASQUERADE
-A POSTROUTING -s 172.30.22.0/24 -m comment --comment "super administrator" -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
COMMIT

客户端生成

客户端配置文件模板

client
dev tap
proto tcp

remote 101.132.33.146 11194

resolv-retry infinite

nobind
persist-key
persist-tun
ca ca.crt
cert sample.crt
key sample.key
remote-cert-tls server
comp-lzo
verb 3

生成客户端证书脚本.

#!/usr/bin/env bash

client=$1
group=$2

if [[ x$client = x || x$group = x ]]; then
    echo "Usage: $0 clientname group"
    exit 1
fi
cd /etc/openvpn/easy-rsa/3/

declare -A groupNetPrefixArr

// 对不同的部门进行分组
// 不同部门的权限不一样. 服务器统一管理
groupNetPrefixArr=(
  ["platform"]="172.30.0."
  ["spider"]="172.30.1."
  ["product"]="172.30.2."
  ["bi"]="172.30.10."
  ["data"]="172.30.100."
)
baseDir="/etc/openvpn"
clientConfigDir="$baseDir/junhsue/users"

_netPrefix=${groupNetPrefixArr[$group]}
netPrefix=${_netPrefix:? error group name!!}

while true
do
  randomNum=$(expr $RANDOM % 255)
  clientIp="${netPrefix}${randomNum}"
  if ! grep -qrn $clientIp $clientConfigDir ;then
      echo "-- user $client assign ip [$clientIp]"
      break
  fi
  echo "-- $clientIp already in use..."
  sleep 1s
done

echo "Generating profiles..."
cat > $clientConfigDir/$client << eof
ifconfig-push $clientIp 255.255.0.0
## >>$group<<
`cat $baseDir/junhsue/$group`
eof
echo "Generating done..."
#exit

if [[ ! -e keys/${client}.key ]]; then
    echo "Generating keys..."
    . vars
    ./easyrsa build-client-full $client nopass
    echo "...keys generated."
fi

tarball=./keys/$client.tgz

if [ ! -e $tarball ]; then
    echo "Creating tarball..."
    tmpdir=/tmp/client-tar.$$
    mkdir $tmpdir
    cp client.ovpn $tmpdir/client.ovpn
    sed -i "s/sample/$client/g" $tmpdir/client.ovpn
    cp keys/ca.crt $tmpdir
    cp pki/private/$client.key $tmpdir/
    cp pki/issued/$client.crt $tmpdir/
    tar -C $tmpdir -czvf $tarball .
    rm -rf $tmpdir
    echo "...tarball created"
else
    echo "Nothing to do, so nothing done. (keys/$client.tgz already exists)"
fi

删除客户端证书脚本

#!/usr/bin/env bash
source vars
cd /etc/openvpn/easy-rsa/3/
client=$1
./easyrsa revoke $client
./easyrsa gen-crl
find ./ -name "$client*" -exec rm -fv {} \;

总结

至此, openvpn的访问权限控制已经完毕. 核心就是利用tap模式的ifconfig-pool来实现客户端的ip受控, 再根据客户端的ip进行分类,分组, 同组成员的权限相同,根据各组的目录权限,push各组的route值客户端主机上, 根据客户端的ip来进行SNAT,实现权限分离. 设置 一组ip, 为超级用户ip组, 拥有超级权限. 访问所有资源. 抽取vpn的公共权限, 推送至每个客户端的路由上. (最低权限.)

参考

• SNAT和MASQUERADE的区别

• tun和tap模式的区别

• Tun/Tap设备基本原理

• centos7安装openvpn

谢谢您的观看