将Pod验证到在gke上运行的外部保管库服务时，权限被拒绝

如何解决将Pod验证到在gke上运行的外部保管库服务时，权限被拒绝

GKE版本-1.14 目前，我有两个私有gke群集（保险柜群集和应用程序群集）

出现以下错误：

vault errors -
auth.kubernetes.auth_kubernetes_b0f01fa6: login unauthorized due to: Post "https://10.V.V.194:443/apis/authentication.k8s.io/v1/tokenreviews": dial tcp `10.V.V.194`:443: i/o timeout

->其中

10.V.V.194 -- is master IP address (no https://) via `kubectl cluster-info

应用程序pod日志

    * permission denied" backoff=1.324573453
2020-10-12T14:39:46.421Z [INFO]  auth.handler: authenticating
2020-10-12T14:40:16.427Z [ERROR] auth.handler: error authenticating: error="Error making API request.

URL: PUT http://10.LB.LB.38:8200/v1/auth/kubernetes/login
Code: 403. Errors:

* permission denied" backoff=2.798763368

->哪里

http://10.LB.LB.38:8200 is Internal LB IP 

保险柜设置

    NAME   TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                         
test-vault  LoadBalancer   240.130.0.59   10.LB.LB.38   8200:32105/TCP,8201:31147/TCP  

K8s身份验证方法如何启用

    $ export VAULT_SA_NAME=$(kubectl get sa vault-auth -o jsonpath="{.secrets[*]['name']}")
$ export SA_JWT_TOKEN=$(kubectl get secret $VAULT_SA_NAME -o jsonpath="{.data.token}" | base64 --decode; echo)
$ export SA_CA_CRT=$(kubectl get secret $VAULT_SA_NAME -o jsonpath="{.data['ca\.crt']}" | base64 --decode; echo) 

# determine Kubernetes master IP address (no https://) via `kubectl cluster-info`
$ export K8S_HOST=<K8S_MASTER_IP>   ----- App cluster ip 

# set VAULT_TOKEN & VAULT_ADDR before next steps
$ vault auth enable kubernetes
$ vault write auth/kubernetes/config \
        token_reviewer_jwt="$SA_JWT_TOKEN" \
        kubernetes_host="https://$K8S_HOST:443" \
        kubernetes_ca_cert="$SA_CA_CRT"

保险库如何在应用程序集群中注入设置

name: AGENT_INJECT_VAULT_ADDR
      value: http://10.LB.LB.38:8200

集群B（应用集群）

kubectl create serviceaccount vault-auth -n default
-----
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: role-tokenreview-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: vault-auth
  namespace: default

vault auth enable kubernetes
-----------
vault write auth/kubernetes/config kubernetes_host="${K8S_HOST}"   
    kubernetes_ca_cert="${VAULT_SA_CA_CRT}" 
    token_reviewer_jwt="${TR_ACCOUNT_TOKEN}"
-----------
vault secrets enable -path=secret/ kv
-----------
vault policy write myapp-kv-rw - <<EOF
 path "secret/myapp/*" {
 capabilities = ["create","read","update","delete","list"]
}
--------------
vault write auth/kubernetes/role/myapp-role \
 bound_service_account_names=default \
 bound_service_account_namespaces=default \
 policies=default,myapp-kv-rw \
 ttl=15m

如果我想念什么，请告诉我吗？

解决方法

您可以尝试使用与设置保险柜相同的配置，从保险柜群集中手动访问Kubernetes API（在您的应用集群中）。

curl -X "POST" "${K8S_HOST}/apis/authentication.k8s.io/v1/tokenreviews" \
     --cacert <(echo $VAULT_SA_CA_CRT)
     -H 'Authorization: Bearer ${TR_ACCOUNT_TOKEN}' \
     -H 'Content-Type: application/json; charset=utf-8' \
     -d $'{
  "kind": "TokenReview","apiVersion": "authentication.k8s.io/v1","spec": {
    "token": "${INTERNAL_APP_TOKEN}"
  }
}'

如果这不起作用，保险柜将无法调用API来审核令牌。如果您的连接超时或被拒绝，则很可能是防火墙问题。如果未经授权，则群集角色设置可能不正确。