Kubernetes搭建spinnaker服务

编程之家收集整理的这篇文章主要介绍了Kubernetes搭建spinnaker服务编程之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

背景:

2017-2018年左右的吧,不记得看什么了看到了spinnaker,但是当时真的安装不起来。各种被墙裂。2020年底学习了泽阳大佬的spinnaker实践课程。通过Halyard方式搭建了spinnaker的集群,并与jenkins gitlab harbor k8s完成了集成。2021年初稍微玩了一下,就去整别的事情去了,没有能应用于线上环境。下半年了,jenkins k8s这些的流程现在基本都是清晰了。想把cd从jenkins中剥离出来教给spinnaker了,就重新温习一下spinnaker吧!

关于spinnaker

spinnaker是Netfix公司开源的一款持续部署工具,采用java语言编写,遵循微服务的设计思想,目标是为团队提供灵活的持续部署流水线并提供软件的部署效率

spinnaker的优势

spinnaker架构

关于spinnaker架构说明

  • deck- 基于浏览器的 UI
  • gate 微服务api网关 Spinnaker UI 和所有 api 调用者通过 Gate 与 Spinnaker 通信
  • orca 流水线阶段编排引擎 它处理所有临时操作和管道。阅读有关 Orca 服务概述的更多信息
  • clouddriver 负责对云提供商的所有变异调用以及索引/缓存所有部署的资源。
  • front50用于持久化应用程序、管道、项目和通知的元数据
  • rosco为各种云提供商生成不可变的 VM 映像(或映像模板)

它用于生成机器映像(例如 GCE 映像AWS AMIAzure VM 映像 )。它目前包装了 packer ,但将被扩展以支持用于生成图像的其他机制。

  • igor用于通过 Jenkins 和 Travis CI 等系统中的持续集成作业触发管道,它允许在管道中使用 Jenkins/Travis 阶段
  • echo 事件总线支持发送通知(例如 Slack、电子邮件、SMS),并对来自 Github 等服务的传入 webhook 采取行动。
  • fiat 认证授权中心 它用于查询用户对帐户、应用程序和服务帐户的访问权限
  • kayenta 自动金丝雀分析
  • Keel管理交付提供动力undefined注:这个还没有用过
  • halyard 配置服务 管理上述每项服务的生命周期。它仅在 Spinnaker 启动、更新和回滚期间与这些服务交互。服务依赖调用关系:

    image.png
    些东西去看官方文档很是详细,比其他的比较详细多了:https://spinnaker.io/docs/reference/architecture/microservices-overview/

Kubernetes搭建spinnaker服务

注:spinnaker的安装方式有helm 和halyard的本地部署方式 这里采用了halyard的方式!。基本过程参照泽阳大佬的spinnaker课程!

本人集群环境为kubernetes1.20.6 rutime使用了containerd并没有采用docker。中间过程尝试了很多次各种失败,先基于docker的方式做一次安装部署。后面剖析一下containerd方式!

基本环境

腾讯云同一vpc内服务器,内网互通,ip为内网地址

主机名

ip

系统

内核

k8s版本

k8s-master-01

10.0.0.41

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-master-02

10.0.0.34

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-master-03

10.0.0.26

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-node-01

10.0.4.49

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-node-02

10.0.4.48

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-node-03

10.0.4.23

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-node-04

10.0.4.47

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-node-05

10.0.4.32

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

containerd

k8s-node-06

10.0.4.18

CentOS Linux 8

5.4.134-1.el8.elrepo.x86_64

v1.21.3

docker

k8s-01

10.0.2.17

CentOS Linux 8

4.18.0-305.12.1.el8_4.x86_64

不在集群内(但是也是一个测试的k8s集群,故上面的其他pod忽略)

docker(集群外一台运行docker的服务器)

注:个人尝试containerd运行halyard未能成功,最终使用docker方式运行halyard

基于docker runtime方式部署halyard的方式部署spinnaker

注: 关于halyard的操作都在k8s-01节点操作。另外声明一下k8s-01原主机名为k8s-02使用了hostnamectl set-hostname修改主机名。有些截图或者命令都依然为k8-02,实际为同一个台服务器。xshell早些时候打开10.0.2.17的窗口......

下载镜像,挂载本地配置文件目录,并启动容器

[root@k8s-01 ~]# docker pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
####创建.hall文件夹后面持久化存储spinnaker生成文件
[root@k8s-01 ~]# mkdir -p /home/spinnaker/.hal
###创建.kube文件夹并将集群中的config文件上传到此目录
[root@k8s-01 ~]# mkdir -p /home/spinnaker/.kube
[root@k8s-01 ~]# ls  /home/spinnaker/.kube
config
####启动halyard容器
[root@k8s-01 ~]# docker run -itd --name halyard   -v /home/spinnaker/.hal:/home/spinnaker/.hal   -v /home/spinnaker/.kube:/home/spinnaker/.kube   registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0

image.png

特权身份进入容器关闭gcs

## 以root身份进入容器,修改配置文件
[root@k8s-01 .kube]# docker exec -it -u root halyard bash
bash-5.0# 
## 修改spinnaker.config.input.gcs.enabled = false 。
vi /opt/halyard/config/halyard.yml
 
spinnaker:
  artifacts:
    debian: https://dl.bintray.com/spinnaker-releases/debians
    docker: gcr.io/spinnaker-marketplace
  config:
    input:
      gcs:
        enabled: false
      writerEnabled: false
      bucket: halconfig

image.png

重新启动halyard容器

## 需要重启容器(如果此命令未重启,则需要退出容器然后 docker restart halyard)
bash-5.0# hal shutdown
Halyard Daemon Response: Shutting down, bye...
##重启容器
[root@k8s-01 .kube]# docker start halyard
halyard

image.png

上传boms文件到服务器

参照https://github.com/zeyangli/spinnaker-cd-install,这里使用的是https://github.com/zeyangli/spinnaker-cd-install/actions/runs/1368350526 1.26.6的制品:

image.png
###通过rz命令上传制品库到运行halyard的服务器,并解压压缩包
[root@k8s-01 work]# ls
1.26.6-Install-Scripts.zip
[root@k8s-01 work]# unzip 1.26.6-Install-Scripts.zip

image.png

嗯看到了这个.boms的文件夹,将其copy到/home/spinnaker/.hal/目录下!

[root@k8s-01 1.26.6]# ls .boms/
bom  clouddriver  deck  echo  fiat  front50  gate  igor  kayenta  monitoring-daemon  orca  rosco
[root@k8s-01 1.26.6]# cp -Ra .boms/ /home/spinnaker/.hal/
[root@k8s-01 1.26.6]# ls /home/spinnaker/.hal/.boms/
bom  clouddriver  deck  echo  fiat  front50  gate  igor  kayenta  monitoring-daemon  orca  rosco

image.png

关于镜像的下载

镜像下载泽阳大佬的制品库下载中有下载镜像的脚本:

#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="node01.zy.com node02.zy.com"

## 下载镜像
function Getimages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$( cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh ${node} "docker pull ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh ${node} "docker tag ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
       ssh ${node} "docker images | grep 'spinnaker-marketplace' "
    done
    
}

Getimages

But 我的集群的运行时是containerd。ctr crictl两个命令的区别有必要重新温习一下。crictl也没法修改标签啊?

#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"

## 下载镜像
function Getimages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$( cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl pull  ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl images ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
       ssh -p 36000 ${node} "crictl images ls| grep 'spinnaker-marketplace' "
    done
    
}

Getimages

所以这个方式就行不通了,然后偶然搜到csdn的---安装篇——用halyard安装Spinnaker。通过在.hall目录下default/service-settings/目录创建对应配置文件。并设置artifactId!

至于service-settings目录为什么在default目录下我也不求甚解泽阳大佬的课程中修改redis为外部redis的时候有这个目录

image.png

image.png
[root@k8s-2 .hal]# mkdir -p /home/spinnaker/.hal/default/service-settings
[root@k8s-2 .hal]# cd /home/spinnaker/.hal/default/service-settings
[root@k8s-2 service-settings]# pwd
/home/spinnaker/.hal/default/service-settings
[root@k8s-2 service-settings]# ls
clouddriver.yml  deck.yml  echo.yml  fiat.yml  front50.yml  gate.yml  igor.yml  kayenta.yml  orca.yml  rosco.yml
[root@k8s-2 service-settings]# cat *
artifactId: docker.io/spinnakercd/clouddriver:8.0.4-20210625060028
artifactId: docker.io/spinnakercd/deck:3.7.2-20210614020020 
artifactId: docker.io/spinnakercd/echo:2.17.1-20210429125836 
artifactId: docker.io/spinnakercd/fiat:1.16.0-20210422230020
artifactId: docker.io/spinnakercd/front50:0.27.1-20210625161956
artifactId: docker.io/spinnakercd/gate:1.22.1-20210603020019
artifactId: docker.io/spinnakercd/igor:1.16.0-20210422230020
artifactId: docker.io/spinnakercd/kayenta:0.21.0-20210322140019 
artifactId: docker.io/spinnakercd/orca:2.20.3-20210630022216
artifactId: docker.io/spinnakercd/rosco:0.25.0-20210422230020 

image.png

image.png

就不修改标签直接使用泽阳大佬docker的镜像仓库里面的镜像了免去下载镜像修改标签的步骤

Halyard配置管理

注: halyard的配置都在k8s-01节点执行认在halyard容器内

设置Spinnaker版本,--version 指定版本

[root@k8s-01 .kube]# docker exec -it -u root halyard bash
bash-5.0$ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
- Edit Spinnaker version
  Failure
Validation in Global:
! ERROR Failure writing your halconfig to path
  "/home/spinnaker/.hal/config": /home/spinnaker/.hal/config

- Failed to update version.

image.png

嗯强调一下 .hall目录要有读写权限啊

[root@k8s-01 1.26.6]# chmod 777 -R /home/spinnaker/.hal/
[root@k8s-01 1.26.6]# 

继续指定spinnaker版本并生成配置文件

bash-5.0$ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
+ Edit Spinnaker version
  Success
+ Spinnaker has been configured to update/install version
  "local:1.26.6". Deploy this version of Spinnaker with `hal deploy apply`.
bash-5.0$ ls
config   default
bash-5.0$ cat config 
currentDeployment: default
deploymentConfigurations:
- name: default
  version: local:1.26.6
  providers:
    appengine:
      enabled: false
      accounts: []
    aws:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: us-west-2
      defaults:
        iamRole: BaseIAMRole
    ecs:
      enabled: false
      accounts: []
    azure:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: azure-linux.json
        baseImages: []
    dcos:
      enabled: false
      accounts: []
      clusters: []
    dockerRegistry:
      enabled: false
      accounts: []
    google:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: gce.json
        baseImages: []
        zone: us-central1-f
        network: default
        useInternalIp: false
    huaweicloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    kubernetes:
      enabled: false
      accounts: []
    tencentcloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    oracle:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: oci.json
        baseImages: []
    cloudfoundry:
      enabled: false
      accounts: []
  deploymentEnvironment:
    size: SMALL
    type: LocalDebian
    imageVariant: SLIM
    updateVersions: true
    consul:
      enabled: false
    vault:
      enabled: false
    customSizing: {}
    sidecars: {}
    initContainers: {}
    hostAliases: {}
    affinity: {}
    tolerations: {}
    nodeselectors: {}
    gitConfig:
      upstreamUser: spinnaker
    livenessProbeConfig:
      enabled: false
    haServices:
      clouddriver:
        enabled: false
        disableClouddriverRoDeck: false
      echo:
        enabled: false
  persistentStorage:
    azs: {}
    gcs:
      rootFolder: front50
    redis: {}
    s3:
      rootFolder: front50
    oracle: {}
  features:
    auth: false
    fiat: false
    chaos: false
    entityTags: false
  metricStores:
    datadog:
      enabled: false
      tags: []
    prometheus:
      enabled: false
      add_source_Metalabels: true
    stackdriver:
      enabled: false
    newrelic:
      enabled: false
      tags: []
    period: 30
    enabled: false
  notifications:
    slack:
      enabled: false
    twilio:
      enabled: false
      baseUrl: https://api.twilio.com/
    github-status:
      enabled: false
  timezone: America/Los_Angeles
  ci:
    jenkins:
      enabled: false
      masters: []
    travis:
      enabled: false
      masters: []
    wercker:
      enabled: false
      masters: []
    concourse:
      enabled: false
      masters: []
    gcb:
      enabled: false
      accounts: []
    codebuild:
      enabled: false
      accounts: []
  repository:
    artifactory:
      enabled: false
      searches: []
  security:
    apiSecurity:
      ssl:
        enabled: false
    uiSecurity:
      ssl:
        enabled: false
    authn:
      oauth2:
        enabled: false
        client: {}
        resource: {}
        userInfoMapping: {}
      saml:
        enabled: false
        userAttributeMapping: {}
      ldap:
        enabled: false
      x509:
        enabled: false
      iap:
        enabled: false
      enabled: false
    authz:
      groupMembership:
        service: EXTERNAL
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
        ldap:
          roleProviderType: LDAP
      enabled: false
  artifacts:
    bitbucket:
      enabled: false
      accounts: []
    gcs:
      enabled: false
      accounts: []
    oracle:
      enabled: false
      accounts: []
    github:
      enabled: false
      accounts: []
    gitlab:
      enabled: false
      accounts: []
    gitrepo:
      enabled: false
      accounts: []
    http:
      enabled: false
      accounts: []
    helm:
      enabled: false
      accounts: []
    s3:
      enabled: false
      accounts: []
    maven:
      enabled: false
      accounts: []
    templates: []
  pubsub:
    enabled: false
    google:
      enabled: false
      pubsubType: GOOGLE
      subscriptions: []
      publishers: []
  canary:
    enabled: false
    serviceIntegrations:
    - name: google
      enabled: false
      accounts: []
      gcsEnabled: false
      stackdriverEnabled: false
    - name: prometheus
      enabled: false
      accounts: []
    - name: datadog
      enabled: false
      accounts: []
    - name: signalfx
      enabled: false
      accounts: []
    - name: aws
      enabled: false
      accounts: []
      s3Enabled: false
    - name: newrelic
      enabled: false
      accounts: []
    reduxLoggerEnabled: true
    defaultJudge: NetflixACAJudge-v1.0
    stagesEnabled: true
    templatesEnabled: true
    showAllConfigsEnabled: true
  spinnaker:
    extensibility:
      plugins: {}
      repositories: {}
  webhook:
    trust:
      enabled: false
  stats:
    enabled: true
    endpoint: https://stats.spinnaker.io
    instanceId: 01FKDR1B3P8PF35RRC93XTE9AS
    deploymentMethod: {}
    connectionTimeoutMillis: 3000
    readTimeoutMillis: 5000
bash-5.0$     
    

设置时区

# 设置时区
hal config edit --timezone Asia/Shanghai

S3--no-validate

# 设置存储为s3(后面不用,但是必须配置bug)
hal config storage edit --type s3  --no-validate

访问方式,设置deck与gate的域名

# 访问方式:设置deck与gate的域名
hal config security ui edit --override-base-url http://spinnaker.xxxx.com
hal config security api edit --override-base-url http://spin-gate.xxxx.com

image.png

来对比一下执行以上命令后config文件的变化:

image.png

image.png

做这些对比是为了方便以后自己手动更改配置文件。大佬的可以忽略这些截图步骤。

添加镜像仓库(harbor)和k8s集群账户

开启镜像仓库配置并添加account

bash-5.0$ hal config provider docker-registry enable --no-validate
+ Get current deployment
  Success
+ Edit the dockerRegistry provider
  Success
+ Successfully enabled dockerRegistry
bash-5.0$ hal config provider docker-registry account add my-harbor-registry \
>     --address https://harbor.xxxx.com \
>     --username xxxx \
>     --password xxxx
+ Get current deployment
  Success
+ Add the my-harbor-registry account
  Success
Validation in
  default.provider.dockerRegistry.my-harbor-registry:
- WARNING Your docker registry has no repositories specified, and
  the registry's catalog is empty. Spinnaker will not be able to deploy any images
  until some are pushed to this registry.
? Manually specify some repositories for this docker registry to
  index.

+ Successfully added account my-harbor-registry for provider
  dockerRegistry.

image.png

开启kubernetes配置并添加account

bash-5.0$ hal config provider kubernetes enable
+ Get current deployment
  Success
+ Edit the kubernetes provider
  Success
Validation in default.provider.kubernetes:
- WARNING Provider kubernetes is enabled, but no accounts have been
  configured.

+ Successfully enabled kubernetes
bash-5.0$ hal config provider kubernetes account add default \
>     --docker-registries my-harbor-registry \
>     --context $(kubectl config current-context) \
>     --service-account true \
>     --omit-namespaces=kube-system,kube-public \
>     --provider-version v2 \
>     --no-validate
+ Get current deployment
  Success
+ Add the default account
  Success
+ Successfully added account default for provider kubernetes.

image.png

再瞄一眼配置文件config:

image.png

image.png

指定部署使用account和命名空间,部署方式distributed(分布式)

bash-5.0$ hal config deploy edit \
>     --account-name default \
>     --type distributed \
>     --location spinnaker 

image.png

看了一眼配置文件应该对应的是deploymentEnvironment下面的配置:

image.png

开启一些主要的功能(后期可以再追加)

bash-5.0$ hal config features edit --pipeline-templates true
bash-5.0$ hal config features edit --artifacts true
bash-5.0$ hal config features edit --managed-pipeline-templates-v2-ui true 

查看config配置文件对应的为features下开关:

image.png

配置与jenkins CI集成

# 配置Jenkins
hal config ci jenkins enable
### JenkinsServer 需要用到账号和密码
hal config ci jenkins master add my-jenkins-master-01 \
    --address https://jenkins.xxxx.com \
    --username zhangpeng \
    --password xxxx
### 启用csrf
hal config ci jenkins master edit my-jenkins-master-01 --csrf true

image.png

cat config对应如下:当然了也可以开启travis wercker consourse gcb等ci工具?

image.png

配置GitHub/GitLab集成

github的是泽阳大佬的。我这里就只集成了gitlab。github仅供参考在配置文件中也生成一下。方便对比配置文件。token的生成就不用做过多的赘述了!

# GitHub
## 参考:https://spinnaker.io/setup/artifacts/github/
## 创建token https://github.com/settings/tokens

hal config artifact github enable

hal config artifact github account add my-github-account \
    --token xxxxxxxxxxxxxxxxxxxxxxx  \
    --username zeyangli

# GitLab
## https://spinnaker.io/setup/artifacts/gitlab/
## 创建一个个人的token(admin)
hal config artifact gitlab enable
hal config artifact gitlab account add my-gitlab-account \
    --token xxxxxxxxxxxxxx

image.png

artifacts下找到相关配置

image.png

使用外部redis集群

关于redis我是使用的腾讯云的云redis。正常该搞一个密码的。但是没有去仔细看下官方文档,就直接使用了免密的方式!

## service-settings
bash-5.0$ pwd
/home/spinnaker/.hal/default/service-settings
vi .hal/default/service-settings/redis.yml

overrideBaseUrl: redis://10.0.0.31:6379
skipLifeCycleManagement: true



## profiles
## /home/spinnaker/.hal/default/profiless
bash-5.0$ pwd
/home/spinnaker/.hal/default
bash-5.0$ mkdir /home/spinnaker/.hal/default/profiles
bash-5.0$ cd profiles/
bash-5.0$ vi gate-local.yml

redis:
    configuration:
         secure:
              true

image.png

image.png

使用sql数据库

MysqL我是直接开启了腾讯云的TDsql-C

image.png

Clouddriver服务

创建数据库

CREATE DATABASE `clouddriver` DEFAULT CHaraCTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

GRANT
  SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW
ON `clouddriver`.*
TO 'clouddriver_service'@'%' IDENTIFIED BY 'clouddriver@spinnaker.com';


GRANT
  SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW
ON `clouddriver`.*
TO 'clouddriver_migrate'@'%' IDENTIFIED BY 'clouddriver@spinnaker.com';

image.png

修改配置文件

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi clouddriver-local.yml
sql:
  enabled: true
  # read-only boolean toggles `SELECT` or `DELETE` health checks for all pools.
  # Especially relevant for clouddriver-ro and clouddriver-ro-deck which can
  # target a sql read replica in their default pools.
  read-only: false
  taskRepository:
    enabled: true
  cache:
    enabled: true
    # These parameters were determined to be optimal via benchmark comparisons
    # in the Netflix production environment with aurora. Setting these too low
    # or high may negatively impact performance. These values may be sub-optimal
    # in some environments.
    readBatchSize: 500
    writeBatchSize: 300
  scheduler:
    enabled: true

  # Enable clouddriver-caching's clean up agent to periodically purge old
  # clusters and accounts. Set to true when using the Kubernetes provider.
  unkNown-agent-cleanup-agent:
    enabled: false

  connectionPools:
    default:
      # additional connection pool parameters are available here,
      # for more detail and to view defaults, see:
      # https://github.com/spinnaker/kork/blob/master/kork-sql/src/main/kotlin/com/netflix/spinnaker/kork/sql/config/ConnectionPoolProperties.kt
      default: true
      jdbcUrl: jdbc:MysqL://10.0.4.22:3306/clouddriver
      user: clouddriver_service
      password: clouddriver@spinnaker.com
    # The following tasks connection pool is optional. At Netflix, clouddriver
    # instances pointed to aurora read replicas have a tasks pool pointed at the
    # master. Instances where the default pool is pointed to the master omit a
    # separate tasks pool.
    tasks:
      user: clouddriver_service
      jdbcUrl: jdbc:MysqL://10.0.4.22:3306/clouddriver
      password: clouddriver@spinnaker.com
  migration:
    user: clouddriver_migrate
    jdbcUrl: jdbc:MysqL://10.0.4.22:3306/clouddriver
    password: clouddriver@spinnaker.com

redis:
  enabled: false
  cache:
    enabled: false
  scheduler:
    enabled: false
  taskRepository:
    enabled: false

Front50服务

创建数据库

CREATE DATABASE `front50` DEFAULT CHaraCTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW ON `front50`.*  TO 'front50_service'@'%' IDENTIFIED BY "front50@spinnaker.com";

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW ON `front50`.* TO 'front50_migrate'@'%' IDENTIFIED BY "front50@spinnaker.com";

image.png

修改配置文件

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi front50-local.yml
spinnaker:
  s3:
    enabled: false
sql:
  enabled: true
  connectionPools:
    default:
      # additional connection pool parameters are available here,
      # for more detail and to view defaults, see:
      # https://github.com/spinnaker/kork/blob/master/kork-sql/src/main/kotlin/com/netflix/spinnaker/kork/sql/config/ConnectionPoolProperties.kt
      default: true
      jdbcUrl: jdbc:MysqL://10.0.4.22:3306/front50
      user: front50_service
      password: front50@spinnaker.com
  migration:
    user: front50_migrate
    jdbcUrl: jdbc:MysqL://10.0.4.22:3306/front50
    password: front50@spinnaker.com

Orca服务

创建数据库

set tx_isolation = 'REPEATABLE-READ';

CREATE SCHEMA `orca` DEFAULT CHaraCTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

GRANT 
SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW
ON `orca`.* 
TO 'orca_service'@'%' IDENTIFIED BY "orca@spinnaker.com" ;

GRANT 
SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW 
ON `orca`.* 
TO 'orca_migrate'@'%'  IDENTIFIED BY "orca@spinnaker.com" ;

image.png

修改配置文件

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi front50-local.yml
bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi orca-local.yml

tasks:
  useManagedServiceAccounts: true
sql:
  enabled: true
  connectionPool:
    jdbcUrl: jdbc:MysqL://10.0.4.22:3306/orca
    user: orca_service
    password: orca@spinnaker.com
    connectionTimeout: 5000
    maxLifetime: 30000
    # MariaDB-specific:
    maxPoolSize: 50
  migration:
    jdbcUrl: jdbc:MysqL://10.0.4.22:3306/orca
    user: orca_migrate
    password: orca@spinnaker.com
# Ensure we're only using sql for accessing execution state
executionRepository:
  sql:
    enabled: true
  redis:
    enabled: false
 
# Reporting on active execution metrics will be handled by sql
monitor:
  activeExecutions:
    redis: false
 
# Use sql for Orca's work queue
# Settings from Netflix and may require adjustment for your environment
# Only validated with AWS aurora MysqL 5.7
# Please PR if you have success with other databases
keiko:
  queue:
    sql:
      enabled: true
    redis:
      enabled: false
 
queue:
  zombieCheck:
    enabled: true
  pendingExecutionService:
    sql:
      enabled: true
    redis:
      enabled: false

部署服务

bash-5.0$ hal deploy apply --no-validate

image.png

image.png

创建Ingress访问web测试

apiVersion: networking.k8s.io/v1
kind: Ingress
Metadata:
  name: spinnaker-service
  namespace: spinnaker
  annotations:
    kubernetes.io/ingress.class: traefik  
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: spinnaker.xxxx.com
    http:
     paths:
     - pathType: Prefix
       path: /
       backend:
          service:
            name:  spin-deck
            port:
              number: 9000
  - host: spin-gate.xxxx.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: spin-gate
            port: 
              number: 8084

image.png

通过web浏览器访问https://spinnaker.xxxx.com/ 如下:

image.png

注:至于为什么访问https呢?因为我的代理是traefik slb上面做了跳转。当然了这里应该根据自己实际的环境出发!

集成ldap:

至于为什么集成ldap呢?账号安全方面考虑了当然是基于,还有其他的各种方式:Google Groups, GitHub Teams, SAML Roles, or LDAP groups。参照:https://spinnaker.io/docs/setup/other_config/security/

关于ldap的安装可以参考Kuberneters 搭建openLDAP

首先登陆web管理页面登陆用户

image.png

创建ou-devops

image.png

image.png

image.png

创建inetorgPerson-zhangpeng

image.png

image.png

image.png

Password设置用户zhangpeng的密码

image.png

Commit确认

image.png

最终如下:

image.png

halyard容器中操作.可能复制命令时候出现异常:Was passed main parameter '    --user-search-base' but no main parameter was defined in your arg class。把代码复制到编辑器处理一下

hal config security authn ldap edit \
--user-search-base 'ou=devops,dc=zy,dc=com' \
--url 'ldap://192.168.1.200:389' \
--user-search-filter 'cn={0}' \
--manager-dn 'cn=admin,dc=zy,dc=com' \
--manager-password '12345678'
hal config security authn ldap enable

image.png
bash-5.0$ cd /home/spinnaker/.hal/
bash-5.0$ pwd
/home/spinnaker/.hal
bash-5.0$ cat config

image.png

web访问如下:怀疑我traefik 强跳搞的

image.png

image.png
bash-5.0$ hal deploy apply --no-validate

image.png
[root@k8s-master-01 ~]# kubectl get pods -n spinnaker

image.png

等待pod起来

image.png

image.png

进入首页

image.png

关于授权

首先登陆ldap web管理页面两个用户组 groupOfUniqueNames yunwenzu devops两个组,根据ldap中组进行授权。

ldap创建用户组与用户

yunweizu-用户zhangpeng

9c5b41423c5ba216e8d585f55d98d7a.png

将zhangpeng用户添加到组中:

image.png

image.png

devop用户组-用户huozhonghao

同理将huozhonghao加入devops组

image.png

image.png

halyard中配置:

开启ldap security 配置。并增加相关配置:

hal config security authz ldap edit \
    --url 'ldap://172.19.252.28:389/dc=xxxx,dc=com' \
    --manager-dn 'cn=admin,dc=xxxx,dc=com' \
    --manager-password 'xxxxxx' \
    --user-dn-pattern 'cn={0}' \
    --group-search-base 'ou=devops' \
    --group-search-filter 'uniqueMember={0}' \
    --group-role-attributes 'cn' \
    --user-search-filter 'cn={0}'
hal config security authz edit --type ldap
hal config security authz enable

image.png

image.png

设置那些用户可以访问集群账户、镜像仓库、应用程序

## 配置yunweizu和group02角色的用户可以使用default这个集群账户
hal config provider kubernetes account edit default \
--add-read-permission yunweizu,group02  \
--add-write-permission yunweizu
  
## 配置yunweizu角色的用户可以使用my-harbor-registry账户
hal config provider docker-registry account edit my-harbor-registry \
    --read-permissions yunweizu \
    --write-permissions yunweizu
##更新部署    
hal deploy apply

注:group2 copy自泽阳大佬的课程笔记。保留了没有什么实际意义。当然了也可以去掉的......

image.png

image.png

登陆spinnaker web尝试:

注:用zhangpeng用户建了一个空白的

devops的用户huozhonghao创建一个空白的applications做下测试

image.png

6a2e9267eeeff7af6df9aa416321ee7.png

image.png

就先只看到这里的权限,警告提示告诉你read会所有用户锁定在此应用程序之外。

具体的权限是跟ldap绑定的那么应该是这样的:

1.在ldap管理页面中, 将用户zhangpeng加入devops组

image.png

2.spinnaker登陆zhangpeng用户新建一个应用,yunweizu 读写可执行,devops组仅仅可读。

image.png
  1. 创建一个新的用户组platform将huozhonghao用户加入

image.png
  1. spinnaker web登陆huozhonghao用户

image.png

嗯 这里也可以看到platform组了 修改一下权限试试,删除一下devops的试试:

image.png

增加platform组权限也是失败因为只有read权限,没有writer权限

image.png

开启管道权限

halyard容器中操作:

bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ cat /home/spinnaker/.hal/default/profiles/orca-local.yml
tasks: 
  useManagedServiceAccounts: true

bash-5.0$ cat ~/.hal/default/profiles/settings-local.js
window.spinnakerSettings.feature.managedServiceAccounts = true;
bash-5.0$ hal deploy apply --no-validate

image.png

注意:orca-local.yml中的开启。我其实在orca服务中早配置上了!

image.png

权限的一些测试

测试一下权限。登陆zhangpeng用户新建一个pipeline zhangpeng

image.png

image.png

可以发现认的kubernetes的default account 并可以保存pipeline

image.png

huozhonghao用户修改zhangpeng pipeline中的Manifest.嗯没有操作权限

image.png

嗯给devops组添加一个read kubernetes account的权限是不是要?否则连account都没有!

image.png
bash-5.0$ hal deploy apply --no-validate
[root@k8s-master-01 develop]# kubectl get pods -n spinnaker

等待clouddriver running!

image.png
[root@k8s-master-01 develop]#kubectl get svc -n spinnaker
[root@k8s-master-01 develop]# curl -X POST http://172.19.254.33:7003/roles/sync
[root@k8s-master-01 develop]#curl 172.19.254.33:7003/authorize/huozhonghao

image.png

read权限依然无法看到accout!

image.png

kubernetes default account 添加devops组writer权限:

image.png
bash-5.0$ vi config 
bash-5.0$ hal deploy apply --no-validate

继续等待clouddriver crunning

image.png

嗯再次刷新web登陆huozhonghao用户可以看到kubernetes default account了但是修改Manifest无法writer。验证通过!

image.png

安装环境基本完成。其他的步骤后续操作

一些失败的尝试(还是没有成功)

1. 下载Halyard 镜像并启动容器---ctr各种命令的复习

ctr pull

[root@k8s-master-01 ~]# ctr image pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
[root@k8s-master-01 ~]# mkdir /root/.hal

image.png

参考一下docker时代的启动方式:

docker run -itd --name halyard \
  -v /root/.hal:/home/spinnaker/.hal \
  -v /root/.kube:/home/spinnaker/.kube \
  registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0

ctr run

依着葫芦画瓢一下?

ctr run -itd --name halyard \
  -v /root/.hal:/home/spinnaker/.hal \
  -v /root/.kube:/home/spinnaker/.kube \
  registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0

中间尝试了很多次各种 ctr命令确实没有搞明白......参考了使用ctr 命令管理 Containerd 容器

我觉得使用containerd安装spinnaker 这真的是可以复习ctr critical命令了

ctr create

[root@k8s-master-01 1.26.6]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw 
[root@k8s-master-01 1.26.6]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2    

ctr t start

[root@k8s-master-01 1.26.6]# ctr t start -d  halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    1729924    RUNNING

image.png

现在问题来了 如何进入容器呢?

ctr tasks exec -t --exec-id

[root@k8s-master-01 1.26.6]# ctr tasks list
TASK       PID        STATUS    
halyard    1729924    RUNNING
[root@k8s-master-01 1.26.6]# ctr tasks exec -t --exec-id 1729924 halyard sh
/ $ 

image.png

image.png

ctr c rm ctr c kill----读写权限没有搞明白 只能采用挂载本地文件的方式重新搞一波了

嗯哼没有权限?docker的时候可以用root的特权模式进入,这里的ctr也没有找到相关命令。然后就偷懒吧halyard.yml文件copy出来:

true修改为false!

image.png

然后挂载文件夹的方式去执行!删除容器重新走一遍流程,走一遍ctr命令

删除容器应该是先停止?stop?结果不出意外我想错了是kill......当然了ctr t kill --signal 9 halyard强制也很重要

[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    4184764    RUNNING
[root@k8s-master-01 1.26.6]# ctr t kill halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    4184764    STOPPED
[root@k8s-master-01 1.26.6]# ctr t ls
TASK       PID        STATUS    
halyard    4184764    STOPPED
[root@k8s-master-01 1.26.6]# ctr c rm halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK    PID    STATUS 

image.png
[root@k8s-master-01 1.26.6]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw 
[root@k8s-master-01 1.26.6]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2    
[root@k8s-master-01 1.26.6] # ctr t start -d  halyard
[root@k8s-master-01 1.26.6] # ctr t ls
TASK       PID        STATUS    
halyard    1729924    RUNNING
[root@k8s-master-01 1.26.6] # ctr tasks exec -t --exec-id 1729924 halyard sh

image.png

下载镜像的尝试:

小伙伴们觉得下载镜像应该用下面哪个脚本?用ctr or crictl呢?最终使用镜像的是要kubernetes....应该是用crictl的。 ctr搞了kubernetes集群应用是发现不了镜像的!

#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"

## 下载镜像
function Getimages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$( cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl pull ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh -p 36000 ${node} "ctr image tag ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
       ssh -p 36000 ${node} "ctr image ls | grep 'spinnaker-marketplace' "
    done
    
}

Getimages
#!/bin/bash

S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"

## 下载镜像
function Getimages(){
    echo -e "\033[43;34m =====GetImg===== \033[0m"

    IMAGES=$( cat tagfile.txt)

    for image in ${IMAGES}
    do
        for node in ${NODES}
        do 
           echo  -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl pull  ${T_REGISTRY}/${image}"
           echo  -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
           ssh -p 36000 ${node} "crictl images ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
        done
    done
    for node in ${NODES}
    do
       echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
       ssh -p 36000 ${node} "crictl images ls| grep 'spinnaker-marketplace' "
    done
    
}

Getimages

当然了还有一个问题就是 crictl 可以更改镜像名字吗?貌似是不可以的...然后此方式就失败了。

各种失败的尝试-containerd下:

[root@k8s-master-01 .boms]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .boms]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2    
[root@k8s-master-01 .boms]# ctr t start -d  halyard
[root@k8s-master-01 .boms]# ctr t ls
TASK       PID        STATUS    
halyard    1775521    RUNNING
[root@k8s-master-01 .boms]# ctr tasks exec -t --exec-id 1729924 halyard sh
/ $ hal config version edit --version local:1.26.6
~ $ cd /home/spinnaker/.hal/
vi config
timezone: America/Los_Angeles  
timezone: Asia/Shanghai

image.png
hal config storage edit --type s3  --no-validate

image.png
hal config security ui edit --override-base-url http://spinnaker.xxxx.com
hal config security api edit --override-base-url http://spin-gate.xxxx.com

image.png

这都tmd怎么会事情.....要疯了

[root@k8s-master-01 .boms]#  ctr t kill --signal 9  halyard
[root@k8s-master-01 .boms]#  ctr c rm halyard

image.png
[root@k8s-master-01 .boms]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .boms]# ctr c ls
CONTAINER    IMAGE                                                           RUNTIME                  
halyard      registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0    io.containerd.runc.v2    
[root@k8s-master-01 .boms]# ctr t start -d  halyard
[root@k8s-master-01 .boms]# ctr t ls
TASK       PID        STATUS    
halyard    1832934   RUNNING
[root@k8s-master-01 .boms]# ctr tasks exec -t --exec-id 1832934 halyard sh
~ $ cd /home/spinnaker/.hal/
~/.hal $ cat config |grep time
  timezone: Asia/Shanghai
  ~/.hal $ cat config |grep s3
    persistentStoreType: s3
    s3:
    s3:
      s3Enabled: true
      
~/.hal $ cat config |grep com
      baseUrl: https://api.twilio.com/
      overrideBaseUrl: http://spin-gate.xxxx.com
      overrideBaseUrl: http://spinnaker.xxxx.com
~/.hal $ hal config provider kubernetes enable
~/.hal $ hal config provider kubernetes account add default \
    --docker-registries my-harbor-registry \
    --context $(kubectl config current-context) \
    --service-account true \
    --omit-namespaces=kube-system,kube-public \
    --provider-version v2 \
    --no-validate

至于这个地方的报错 他还是需要w 宿主机 chmod了一下

image.png
hal config deploy edit \
    --account-name default \
    --type distributed \
    --location spinnaker 

image.png
hal config features edit --pipeline-templates true
hal config features edit --artifacts true
hal config features edit --managed-pipeline-templates-v2-ui true  

image.png

尼玛又疯了!。。。。。。。。。。。。。。。。。分隔符吧 我准备全部都修改好了这些文件

image.png

我又开始怀疑了 一下人生:是不是我的服务器资源不够了?因为我这是kubernetes的master节点,然后呢资源只有4核心8g,我找一个资源多的server测试一下?

copy一下 .kube下的config

[root@k8s-node-01 home]# mkdir -p /home/spinnaker/.hal
[root@k8s-node-01 home]# mkdir -p /opt/halyard/config
[root@k8s-node-01 home]# mkdir -p /home/spinnaker/.kube
[root@k8s-node-01 home]# crictl pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
Image is up to date for sha256:8673f1670b8768138cd8349b7d9843eb4fd451658227d2e9f02d5fbe454c500d
[root@k8s-node-01 home]# cd /home/spinnaker/.kube
[root@k8s-node-01 .kube]# rz

[root@k8s-node-01 .kube]# ls
config
[root@k8s-node-01 .kube]# ctr image pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
[root@k8s-node-01 .kube]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/home/spinnaker/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw

image.png
[root@k8s-node-01 .boms]# pwd
/home/spinnaker/.hal/.boms
[root@k8s-node-01 .boms]# ls
bom  clouddriver  deck  echo  fiat  front50  gate  igor  kayenta  monitoring-daemon  orca  rosco
[root@k8s-node-01 .boms]# cd /opt/halyard/config/
[root@k8s-node-01 config]# cat halyard.yaml

image.png
[root@k8s-node-01 ~]# ctr t ls
TASK    PID    STATUS    
[root@k8s-node-01 ~]# ctr t start -d  halyard
[root@k8s-node-01 ~]# ctr t ls
TASK       PID        STATUS    
halyard    3910255    RUNNING
[root@k8s-node-01 ~]# ctr tasks exec -t --exec-id 3910255 halyard sh
/ $ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
- Edit Spinnaker version
  Failure
Validation in Global:
! ERROR Failure writing your halconfig to path
  "/home/spinnaker/.hal/config": /home/spinnaker/.hal/config

- Failed to update version.
/ $ hal config version edit --version local:1.26.6
+ Get current deployment
  Success
+ Edit Spinnaker version
  Success
+ Spinnaker has been configured to update/install version
  "local:1.26.6". Deploy this version of Spinnaker with `hal deploy apply`.
/ $ hal config edit --timezone Asia/Shanghai
********又tmd  sb了 不知道怎么回事不试了。直接改好配置文件直接启动了!

总结以上失败 执行啥也不行...最后决定直接把docker环境面config文件以及其他制品搞过来试试!

my config文件

currentDeployment: default
deploymentConfigurations:
- name: default
  version: local:1.26.6
  providers:
    appengine:
      enabled: false
      accounts: []
    aws:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: us-west-2
      defaults:
        iamRole: BaseIAMRole
    ecs:
      enabled: false
      accounts: []
    azure:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: azure-linux.json
        baseImages: []
    dcos:
      enabled: false
      accounts: []
      clusters: []
    dockerRegistry:
      enabled: true
      accounts:
      - name: my-harbor-registry
        requiredGroupMembership: []
        providerVersion: V1
        permissions:
          READ:
          - yunweizu
          WRITE:
          - yunweizu
        address: https://harbor.xxxx.com
        username: zhangpeng
        password: xxxx
        email: fake.email@spinnaker.io
        cacheIntervalSeconds: 30
        clientTimeoutMillis: 60000
        cacheThreads: 1
        paginateSize: 100
        sortTagsByDate: false
        trackDigests: false
        insecureRegistry: false
        repositories: []
      primaryAccount: my-harbor-registry
    google:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: gce.json
        baseImages: []
        zone: us-central1-f
        network: default
        useInternalIp: false
    huaweicloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    kubernetes:
      enabled: true
      accounts:
      - name: default
        requiredGroupMembership: []
        providerVersion: V2
        permissions:
          READ:
          - yunweizu,group02 
          - devops
          WRITE:
          - yunweizu
          - devops
        dockerRegistries:
        - accountName: my-harbor-registry
          namespaces: []
        context: kubernetes-admin@kubernetes
        configureImagePullSecrets: true
        serviceAccount: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces:
        - kube-system
        - kube-public
        kinds: []
        omitKinds: []
        customresources: []
        cachingPolicies: []
        oAuthScopes: []
        onlySpinnakerManaged: false
      primaryAccount: default
    tencentcloud:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
    oracle:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: oci.json
        baseImages: []
    cloudfoundry:
      enabled: false
      accounts: []
  deploymentEnvironment:
    size: SMALL
    type: distributed
    accountName: default
    imageVariant: SLIM
    updateVersions: true
    consul:
      enabled: false
    vault:
      enabled: false
    location: spinnaker
    customSizing: {}
    sidecars: {}
    initContainers: {}
    hostAliases: {}
    affinity: {}
    tolerations: {}
    nodeselectors: {}
    gitConfig:
      upstreamUser: spinnaker
    livenessProbeConfig:
      enabled: false
    haServices:
      clouddriver:
        enabled: false
        disableClouddriverRoDeck: false
      echo:
        enabled: false
  persistentStorage:
    persistentStoreType: s3
    azs: {}
    gcs:
      rootFolder: front50
    redis: {}
    s3:
      rootFolder: front50
    oracle: {}
  features:
    auth: false
    fiat: false
    chaos: false
    entityTags: false
    pipelineTemplates: true
    artifacts: true
    managedPipelineTemplatesV2UI: true
  metricStores:
    datadog:
      enabled: false
      tags: []
    prometheus:
      enabled: false
      add_source_Metalabels: true
    stackdriver:
      enabled: false
    newrelic:
      enabled: false
      tags: []
    period: 30
    enabled: false
  notifications:
    slack:
      enabled: false
    twilio:
      enabled: false
      baseUrl: https://api.twilio.com/
    github-status:
      enabled: false
  timezone: Asia/Shanghai
  ci:
    jenkins:
      enabled: true
      masters:
      - name: my-jenkins-master-01
        permissions: {}
        address: https://jenkins.xxxx.com
        username: zhangpeng
        password: xxxxx
        csrf: true
    travis:
      enabled: false
      masters: []
    wercker:
      enabled: false
      masters: []
    concourse:
      enabled: false
      masters: []
    gcb:
      enabled: false
      accounts: []
    codebuild:
      enabled: false
      accounts: []
  repository:
    artifactory:
      enabled: false
      searches: []
  security:
    apiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: https://spin-gate.xxxx.com
    uiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: https://spinnaker.xxxx.com
    authn:
      oauth2:
        enabled: false
        client: {}
        resource: {}
        userInfoMapping: {}
      saml:
        enabled: false
        userAttributeMapping: {}
      ldap:
        enabled: true
        url: ldap://172.19.252.28:389
        userSearchBase: ou=devops,dc=xxxx,dc=com
        userSearchFilter: cn={0}
        managerDn: cn=admin,dc=xxxx,dc=com
        managerPassword: xxxx
      x509:
        enabled: false
      iap:
        enabled: false
      enabled: true
    authz:
      groupMembership:
        service: LDAP
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
          path: /home/spinnaker/.hal/userrole.yml
        ldap:
          roleProviderType: LDAP
          url: ldap://172.19.252.28:389/dc=xxxx,dc=com
          managerDn: cn=admin,dc=xxxx,dc=com
          managerPassword: xxxx
          userDnPattern: cn={0}
          groupSearchBase: ou=devops
          userSearchFilter: cn={0}
          groupSearchFilter: uniqueMember={0}
          groupRoleAttributes: cn
      enabled: true
  artifacts:
    bitbucket:
      enabled: false
      accounts: []
    gcs:
      enabled: false
      accounts: []
    oracle:
      enabled: false
      accounts: []
    github:
      enabled: true
      accounts:
      - name: my-github-account
        username: zeyangli
        token: xxxx
    gitlab:
      enabled: true
      accounts:
      - name: my-gitlab-account
        token: xxxx
    gitrepo:
      enabled: false
      accounts: []
    http:
      enabled: false
      accounts: []
    helm:
      enabled: false
      accounts: []
    s3:
      enabled: false
      accounts: []
    maven:
      enabled: false
      accounts: []
    templates: []
  pubsub:
    enabled: false
    google:
      enabled: false
      pubsubType: GOOGLE
      subscriptions: []
      publishers: []
  canary:
    enabled: false
    serviceIntegrations:
    - name: google
      enabled: false
      accounts: []
      gcsEnabled: false
      stackdriverEnabled: false
    - name: prometheus
      enabled: false
      accounts: []
    - name: datadog
      enabled: false
      accounts: []
    - name: signalfx
      enabled: false
      accounts: []
    - name: aws
      enabled: false
      accounts: []
      s3Enabled: false
    - name: newrelic
      enabled: false
      accounts: []
    reduxLoggerEnabled: true
    defaultJudge: NetflixACAJudge-v1.0
    stagesEnabled: true
    templatesEnabled: true
    showAllConfigsEnabled: true
  spinnaker:
    extensibility:
      plugins: {}
      repositories: {}
  webhook:
    trust:
      enabled: false
  stats:
    enabled: true
    endpoint: https://stats.spinnaker.io
    instanceId: 01FKDR1B3P8PF35RRC93XTE9AS
    deploymentMethod: {}
    connectionTimeoutMillis: 3000
    readTimeoutMillis: 5000

直接搞过来试一波

image.png

上传文件并解压到k8s-master-01节点home目录下

image.png

继续

[root@k8s-master-01 .kube]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw 
[root@k8s-master-01 .kube]#  ctr t start -d  halyard
[root@k8s-master-01 .kube]# ctr t ls
TASK       PID        STATUS    
halyard    3073271    RUNNING
[root@k8s-master-01 .kube]# ctr tasks exec -t --exec-id 3073271 halyard sh
bash-5.0$  hal deploy apply --no-validate

image.png

重新来一遍

[root@k8s-master-01 .kube]# ctr t kill --signal 9 halyard
[root@k8s-master-01 .kube]# ctr c rm halyard

image.png
[root@k8s-master-01 .hal]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard   --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row   --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .hal]# ctr t start -d  halyard
[root@k8s-master-01 .hal]# ctr t ls
TASK       PID        STATUS    
halyard    3085723    RUNNING
[root@k8s-master-01 .hal]# ctr tasks exec -t --exec-id 3085723 halyard bash
bash-5.0$ 

算了我放弃了......,containerd的安装方式

总结一下失败以及经验:

  1. containerd or docker的运行时中都可以在文件夹 /home/spinnaker/.hal/default/service-settings本地写文件的件方式指定image tag,docker环境下还好,containerd方式下crictl 修改镜像标签自己掌握的不是很好!
  2. containerd命令跟docker还是不一样。启动halyard的方式还是很不好弄,最好的方式还是在一台安装docker的机器上面运行halyard。
  3. halyard执行脚本复制命令的空格格式问题
  4. 部署过程中出现数据库地址写错问题...写成了TDsql-C中的读地址....

总结

以上是编程之家为你收集整理的Kubernetes搭建spinnaker服务全部内容。

如果觉得编程之家网站内容还不错,欢迎将编程之家网站推荐给好友。

原文地址:https://cloud.tencent.com/developer/article/1897838

例如:随着人工智能的不断发展,机器学习这门技术也越来越重要,很多人都开启了学习机器学习,本文就介绍了机器学习的基础内容。提示:以下是本篇文章正文内容,下面案例可供参考例如:以上就是今天要讲的内容,本文仅仅简单介绍了pandas的使用,而pandas提供了大量能使我们快速便捷地处理数据的函数和方法。
failed to find plugin “flannel” in path [/opt/cni/bin],k8sNotReady解决方案
2、k8s-master运行的组件查看,控制平面(官网说法)查看构成k8s主节点。查看 k8s集群状态,查看有哪些工作节点此时node机器就可以和master机器 通信 了,走kubelet进程该进程是以什么形式运行的?宿主机直接,以1号进程,systemd去启动的kubelet进程5、k8s-master主节点,查看所有工作节点的信息6、如何让集群就绪呢?部署网络插件确保集群所有节点,就绪状态11、发出pod创建请求,运行一个nginx-pod访问pod-ip即可pod的ip是k
k8s 基础组件总结
Helm 是 Deis 开发的一个用于 Kubernetes 应用的包管理工具,主要用来管理 Charts。类似于:Ubuntu 中的 APTCentOS 中的 YUM主要功能:创建:创建新的charts打包:将charts打包成tgz文件交互:与chart仓库交互安装:安装和卸载K8s的应用管理:管理使用Helm安装的charts的生命周期
这些天我听到越来越多的关于 Kubernetes 发行版的讨论。在高层次上,我知道人们使用这个术语时的意思。但是,当您环顾生态系统时,有时很难弄清楚 Kubernetes 发行版的确切含义,因为它们有多种形式。它不是什么定义 Kubernetes 发行版不是什么很容易:它不是“普通”Kubernetes,意思是您通过从 GitHub 下载 Kubernetes 源代码、编译并自行安装来创建的 Kubernetes安装。几乎没有人会以这种方式安装 Kubernetes,因为这会花费太多的工作。
目录序言1.基本介绍1.1 命令格式介绍2 基础命令2.1 create2.2 delete2.2.1 根据yaml删除资源2.2.1 根据名称删除资源2.3 get2.3.1查看pod列表2.3.2 查看node2.3.3 查看svc2.3.4 查看all2.3.5 查看ns2.3.4 查看deploy2.3 run 2.4 explain2.5 edit3.故障命令3.1 describe3.2 logs3.2.1 查看容器快照3.2.2 查看已停止的容器3.3 exec 2.2 delet