[转载]Cloudfoundry对接华为公有云部署指南
华为公有云是支持openstack接口的公有云，因此我们可以直接用原生的Cloudfoundry对接OpenStack的方案在华为云上进行CF部署。
部署架构图如下图所示，安装人员需要一台普通的VM主机（对应图中蓝色BOSH CLI），用于控制全局。这个机器就是我们用于管理整个cloudfoundry的Linux机器。该主机可以是Centos7或者Ubuntu16.4，后续我们以国内常用的Ubuntu16.4主机作为范本进行安装部署。
bosh官方对接openstack部署cf文档详见[https://bosh.io/docs/init-openstack/]
1. 准备运行环境
ECS VM | ubuntu 16.04 |
VPC 包括三个网络 network_id1="a95cd147-689c-483a-90ca-dae8c2ed938a " network_id2="2acd71a7-4cdc-4472-a3f4-86438ad2521b" network_id3="f57eec08-4e7a-4375-9783-339c937e4f22" | 用作整个 Bosh 和 Cloud Foundry 的网络运行环境 Network1: 10.0.1.0/24 |
Private ip | 10.0.1.51 |
Security Group | 为网络环境设置访问权限 |
EIP | 与bosh director绑定，提供公网 IP，用于登录bosh director进行后续cf的部署 160.44.206.37 |
ruby | 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14] |
bosh | bosh-cli-3.0.1-linux-amd64 |
Cloudfoundry cli | cf-cli_6.33.0_linux_x86-64 |
Directory vm | Eip: 160.44.206.37 Private ip: 10.0.1.10 |
如下运行环境均可以手动创建，也可以使用下面介绍的terraform工具进行自动创建
1.1.准备一台ubuntu 16.04的执行机，用以安装bosh cli和cloudfoundry cli执行部署cf的命令，以及后面部署cf成功后调用cf命令在cf上部署应用，登录这台执行机进行1.2步骤的操作
1.2.使用terraform创建安装bosh需要的公有云资源
terraform模板参考：https://github.com/cloudfoundry-incubator/bosh-openstack-environment-templates/tree/master/bosh-init-tf
该模板在公有云上会创建的资源如下：
VPC（1个）
Security Group（1个）
EIP（1个）
1.2.1 下载创建bosh需要的公有云资源模板
$ git clone https://github.com/cloudfoundry-incubator/bosh-openstack-environment-templates $ cd bosh-openstack-environment-templates/bosh-init-tf/ $ ./generate_ssh_keypair.sh //生成bosh.pem秘钥，用于登录后续cf相关的vm机器 $ cp terraform.tfvars.template terraform.tfvars |
1.2.2 配置华为云信息
修改配置文件中的值为自己公有云上的信息,如何获取？在华为云右上角点击你自己的用户名->基本信息->管理我的凭证(My Credential)中获取domain_name，project_name(也就是tenant_name)和project_id,domain_name,user_name。如果是中文没有明确的domain字样，可以点击左下角切换到英文。在华为云endpoints列表中获取AZ信息
$ vi terraform.tfvars auth_url = "https://iam.cn-south-1.myhwclouds.com:443/v3" domain_name = "domain_name" user_name = "openstack_user" password = "openstack_password" tenant_name = "cn-south-1" region_name = "cn-south-1" availability_zone = "cn-south-1a"
ext_net_name = "admin_external_e" //在huawei公有云上该值为固定值 ext_net_id = "0a2228f2-7f8a-45f1-8e09-9039e1d09975" //在huawei公有云上该值为固定值
# in case your OpenStack needs custom nameservers # dns_nameservers = 8.8.8.8，100.125.4.25（your_own_system_private_ip） //如果后续cf的出口要用私有域名，那这的dns服务器地址一定要配置成私有dns服务器对应的dns ip， //否则后面的私有域名无法解析，也就无法被访问，也就会导致登录不上 |
1.2.2 配置好以后下载terraform二进制执行文件，运行terraform命令创建资源
$ wget https://releases.hashicorp.com/terraform/0.10.7/terraform_0.10.7_linux_amd64.zip $ unzip terraform_0.10.7_linux_amd64. ip $ ./terraform init //初始化terraform配 $ ./terraform apply //使用terraform创建资源 ... Apply complete! Resources: 11 added, 0 changed, 0 destroyed.
Outputs:
default_key_name = bosh default_security_groups = [bosh] external_ip = 160.44.206.37 internal_cidr = 10.0.1.0/24 internal_gw = 10.0.1.1 internal_ip = 10.0.1.10 net_dns = [8.8.8.8] net_id = a95cd147-689c-483a-90ca-dae8c2ed938a router_id = bdc24a70-6a56-485e-a733-15612925759b |
注： 创建成功以后要记录好回显的信息，作为后面的bosh director的创建的参数入
1.2.3 （Option）如果配置有问题，或者想清理已经创建的数据可以使用如下命令进行清理
$./terraform destroy |
2. 安装bosh director
2.1 以root用户登录到第一步创建的ubuntu执行机器上
$ apt-get update $ sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3 $ ruby –v ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14] |
2.2安装bosh cli
$ wget https://s3.amazonaws.com/bosh-cli-artifacts/bosh-cli-3.0.1-linux-amd64 $ chmod +x bosh-cli-3.0.1-linux-amd64 $ sudo mv ~/bosh-cli-3.0.1-linux-amd64 /usr/local/bin/bosh $ bosh -v version 3.0.1-712bfd7-2018-03-13T23:26:43Z Succeeded |
2.3创建director，也就是刚开始部署图里面绿色方框部分
$ cd /root $ mkdir bosh-1 && cd bosh-1 $ git clone https://github.com/cloudfoundry/bosh-deployment // 修改虚拟机flavor类型为公有云支持的类型s2.large.2 $ vi bosh-deployment/openstack/cpi.yml - type: replace path: /resource_pools/name=vms/cloud_properties? value: instance_type: s2.large.2 availability_zone: ((az)) // 修改虚拟机flavor类型为公有云支持的类型s2.large.2、s2.large.8 $ vi bosh-deployment/openstack/cloud-config.yml vm_types: - name: default cloud_properties: instance_type: s2.large.2 - name: large cloud_properties: instance_type: s2.large.8
$ bosh create-env bosh-deployment/bosh.yml \ --state=state.json \ --vars-store=creds.yml \ -o bosh-deployment/openstack/cpi.yml \ -o bosh-deployment/external-ip-with-registry-not-recommended.yml \ -v director_name=bosh-1 \ -v internal_cidr=10.0.1.0/24 \ -v internal_gw=10.0.1.1 \ -v internal_ip=10.0.1.10 \ -v external_ip=160.44.206.37 \ -v auth_url=https://iam.cn-south-1.myhwclouds.com:443/v3 \ -v default_key_name=bosh \ -v default_security_groups=[bosh] \ -v net_id=a95cd147-689c-483a-90ca-dae8c2ed938a \ -v openstack_password=password \ -v openstack_username=cloud_user \ -v openstack_domain=cloud_domamin \ -v openstack_project=project_name \ -v private_key=./bosh.pem \ -v az=cn-south-1a \ -v region=cn-south-1 |
注:如果包下不下来，可以自己在本地下载后上传到执行机中，并把bosh-deployment/openstack/cpi.yml文件
vi bosh-deployment/openstack/cpiy l 中的相应包路径进行修改, -v state_timeout=30000
-v openstack_flavor=s2.large.2 \ 上传镜像超时设置，和创建虚拟机时候的flavor虚拟机规格设置在bosh cli
v2中也没有生效，需要手动在bosh-deployment/openstack/cpi.yml文件文档中添加或者修改
- type: replace path: /instance_groups/name=bosh/properties/openstack? value: &openstack auth_url: ((auth_url)) username: ((openstack_username)) api_key: ((openstack_password)) domain: ((openstack_domain)) project: ((openstack_project)) region: ((region)) default_key_name: ((default_key_name)) default_security_groups: ((default_security_groups)) state_timeout: 30000 human_readable_vm_names: true |
2.4登录bosh director
$export BOSH_ENVIRONMENT=160.44.206.37 # Configure local alias $ bosh alias-env bosh-1 -e 119.3.21.3 --ca-cert <(bosh int ./creds.yml --path /director_ssl/ca)
# Log in to the Director $ export BOSH_CLIENT=admin $ export BOSH_CLIENT_SECRET=`bosh int ./creds.yml --path /admin_password` $ bosh -e bosh-1 l //登录bosh director Using environment '119.3.21.3'
Using environment '119.3.21.3' as client 'admin'
Logged in to '119.3.21.3'
Succeeed $ bosh envs |
登录bosh director方法2
$ bosh int creds.yml --path /jumpbox_ssh/private_key > jumpbox.key $ chmod 600 jumpbox.key $ ssh jumpbox@external-or-internal-ip -i jumpbox.key |
3. 安装cloudfoundry
3.1 安装cf cli
$ wget -c "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" -O cf-cli_6.33.0_linux_x86-64.tgz $ tar -xzvf cf-cli_6.33.0_linux_x86-64.tgz -C /usr/local/bin $ cf -v cf version 6.36.1+e3799ad7e.2018-04-04 |
3.2 使用cf-deployment进行部署
3.2.1再次使用terraform创建安装cf的时候需要的共有云资源
将 terraform工程 https://github.com/cloudfoundry-incubator/bosh-openstack-environment-templates/tree/master/cf-deployment-tf 下载到执行机上面，配置好terraform全局变量，运行如下命令创建cf所需资源
$ terraform init <cloned-repo-path>/cf-deployment-tf $ terraform apply <cloned-repo-path>/cf-deployment-tf |
创建完成后注意查看回显信息，回显信息中有下面步骤中所需要的网络信息，包括在同一个VPC下创建的三个不同网段的子网信息。
3.2.2下载cf-deployment工程
git clone https://github.com/cloudfoundry/cf-deployment.git |
注：也可以下载 cf-deployment的历史版本https://github.com/cloudfoundry/cf-deployment/releases
3.2.3 修改instance_type为公有云自己的instance_type。修改文件为iaas-support/openstack/cloud-config.yml
3.2.4 上传stemcell镜像文件
cd /root/bosh-1/ wget https://s3.amazonaws.com/bosh-core-stemcells/openstack/bosh-stemcell-3541.10-openstack-kvm-ubuntu-trusty-go_agent.tgz bosh upload-stemcell bosh-stemcell-3541.10-openstack-kvm-ubuntu-trusty-go_agent.tgz |
3.2.5 指定cf deployment的相关配置信息，包括AZ域，子网信息为3.1创建的子网信息。
cd /root/bosh-1 bosh update-cloud-config \ -v availability_zone1="cn-south-1a" \ -v availability_zone2="cn-south-1a" \ -v availability_zone3="cn-south-1a \ -v network_id1="a95cd147-689c-483a-90ca-dae8c2ed938a" \ -v network_id2="2acd71a7-4cdc-4472-a3f4-86438ad2521b" \ -v network_id3="f57eec08-4e7a-4375-9783-339c937e4f22" \ cf-deployment/iaas-support/openstack/cloud-config.yml |
3.2.6 部署cloudfoundry
方案一：以下为部署带loadbalance服务的cf方案
bosh -d cf deploy cf-deployment/cf-deployment.yml \ -o cf-deployment/operations/use-compiled-releases.yml \ -o cf-deployment/operations/openstack.yml \ --vars-store cf-vars.yml \ -v system_domain="example.com" |
方案二：使用haproxy方案，该方案不用装loadbalance资源
https://bosh.io/docs/cloud-config/
在/root/bosh-1/cf-deployment/iaas-support/openstack/cloud-config.yml文件中
添加haproxy的私有ip为static ip到你的网络中
- az: z1 range: 10.0.1.0/20 reserved: [10.0.1.2-10.0.1.50] gateway: 10.0.1.1 static: [10.0.1.51] cloud_properties: net_id: ((network_id1)) security_groups: [cf] |
跟loadbalancer方案不一样的地方是需要添加一个配置文件use-haproxy.yml，已经haproxy用到的private ip（10.0.1.51），该ip可以是在你network id1网段以内没有使用的任意一个私有ip。
bosh -e bosh-1 -d openstack-cf deploy cf-deployment/cf-deployment.yml \ --vars-store cf-vars.yml \ -v system_domain=example.com \ -v haproxy_private_ip=10.0.1.51 \ -o cf-deployment/operations/openstack.yml \ -o cf-deployment/operations/use-haproxy.yml |
4.登录cf
cf login -a https://api.example.com --skip-ssl-validation -u admin -p `bosh int ./cf-vars.yml --path /cf_admin_password` API endpoint: https://api.example.com
Email> admin
Password> Authenticating... OK
Targeted org mycloud
API endpoint: https://api.example.com (API version: 2.51.0) User: admin Org: mycloud Space: No space targeted, use 'cf target -s SPACE' |
5 部署应用
在部署应用时，如果cf需要下载关联包，那么需要cf的vm主机能上互联网，默认主机无法上网，需要申请华为NAT网关服务，把所有网络子网全部加入到SNAT中，并统一通过该NAT上互联网
5.1 创建并指定空间
默认创建名为mycloud的组织org，以及名为development的space空间，一个org组织下可以包含多个空间，每个空间下可以部署多个应用
$ cf create-space development Creating space development in org mycloud as admin... OK Assigning role RoleSpaceManager to user admin in org mycloud / space development as admin... OK Assigning role RoleSpaceDeveloper to user admin in org mycloud / space development as admin... OK
TIP: Use 'cf target -o "mycloud" -s "development"' to target new space $ cf target -o "mycloud" -s "development" api endpoint: https://api.example.com api version: 2.51.0 user: admin org: mycloud space: development |
5.2 下载示例应用demo
$ git clone https://github.com/cloudfoundry-samples/cf-php-demo |
5.3 修改 manifest.yml文件
其中域名为自己的域名，与部署cf时填写的域名一致，这里为example.com
$ cd cf-php-demo/ $ vi manifest.yml --- applications: - name: cf-php-demo memory: 128M instances: 1 host: cf-php-demo domain: example.com path: . buildpack: https://github.com/dmikusa-pivotal/cf-php-apache-buildpack.git |
5.4推送应用
cf push myapp -b php_buildpack |
注：如果push应用的时候仍然报错包下载问题，可进行如下配置国外代理代理解决cf里面vm不能上网导致无法安装的问题。建议尽可能外部编译好再上传应用。
cf set-env myapp http_proxy "http://user:password@ip" cf set-env myapp https_proxy "http://user:password@ip" cf set-env myapp no_proxy "172.0.0.0/8,localhost,192.168.0.0/16,10.0.0.0/8,122.112.204.189" |
也可以把代理配置直接写入manifest.yml文件：
vi manifest.yml ---applications: - name: cf-php-demo memory: 128M instances: 1 host: cf-php-demo path: . env: http_proxy: http://7.90.3.13:250 https_proxy: http://7.90.3.38:250 no_proxy: 172.0..,localhost,127.0.0.1,10.0..,.hwclouds-dns.com,.novalocal,.example.com |
- 点赞
- 收藏
- 关注作者
评论（0）