Terraform 集成Ansible 实战(remote)

语言: CN / TW / HK
时间 2022-07-03 20:15:23

前言

Terraform 和 Ansible 是两个独立的工具，有各自的用途，但它们可以集成以解决典型用例以及它们相互补充的方式，这使得它们更加受欢迎。

一 工具简介

1.1 Terraform

它是作为代码(IaC)软件工具的基础设施，用于构建、更改和版本控制基础设施。它与500多个供应商合作，这些供应商的资源可用于提供基础设施。

1.2 Ansible

它是一种配置管理工具，可在已设置的基础架构上配置和部署应用程序时派上用场。在本文中，我们使用 Ansible 配置了 terraform 部署的基础设施。

本文将使用Terraform对腾讯云基础设施资源进行编排管理，使用Ansible对主机实例进行配置管理。

二 架构及流程

总体方案使用terraform来进行云上基础设施编排，使用ansible进行主机或其他设施的配置管理。

编排出一台独立的远程ansible manager 主机作为ansible 配置管理的管理服务器，需要后续在其上进行ansible 配置、密钥下发，及生成管控目标服务器的invertory主机清单，对需要管控的node节点需要在该服务器进行操作。

三 实战

3.1 编排Ansible manager节点

利用本地密钥编排ansible节点

  • 安装ansible
  • 将ansible的配置文件传输到ansible manger的特定目录

```shell locals { cidr_block_vpc_name = "10.0.0.0/16" cidr_block_subnet_name = "10.0.1.0/24" count = 1 username = "root" local_key_file = "id_rsa" local_ansible_config = "ansible.cfg" ansible_dir = "/opt/ansibleworkspace/" key_file = "/Users/xuel/.ssh/id_rsa" }

data "tencentcloud_images" "image" { image_type = ["PUBLIC_IMAGE"] os_name = "centos 7.5" }

data "tencentcloud_instance_types" "instanceType" { cpu_core_count = 1 memory_size = 1 filter { name = "instance-family" values = ["S3"] } }

data "tencentcloud_availability_zones" "zone" {}

resource "tencentcloud_vpc" "vpc" { cidr_block = local.cidr_block_vpc_name name = "xuel_tf_vpc" }

resource "tencentcloud_subnet" "subnet" { availability_zone = data.tencentcloud_availability_zones.zone.zones.0.name cidr_block = local.cidr_block_subnet_name name = "xuel_tf_subnet" vpc_id = tencentcloud_vpc.vpc.id }

resource "tencentcloud_key_pair" "xuel-tf-key" { key_name = "xuel_tf_ansible_manager" public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCcJxzfjqQDikRRShExu22kvNOfMKeWrn++s5nQMTm+9SNsQISEk+P15JhFVachgjumupMcOIrfJQAAQcnzNmxoRCTCJQfmegGpDZVpE1cyHUhGMA5kwu67PmK1Snm8hkg2Xzlduhr1xysL2mRn3+6o5tsFXhGrYOcSSXnf5SpTPgMjqo339ksH0iv8kvu3NaZRueygLYaVEMjixJvsnUisL3uY8LQ+4cm2Zu5mdQamhWhN0kkSdlfbjPgzxexL4AglD9YDy4I9Q80vKzy33Ubwo17a2aNCF3uPpYvCKiV0H9z2XtMxisKDfsQQA01Q1vpccUIK6L48xSbersxxxxxxxxxxxxxxxxxi1SGabjYLsv23ki6EMGjM/AK+fq+vj3pIPUMpscX3xVDGmz/zusq6v1KfOtQw7B/Dg8c2cxKUlEWZqqC3A7rt3JO/RVEbeqSe5mlRm2yngINVemmhkcfZNs= [email protected]" }

resource "tencentcloud_instance" "xuel_tf_ansible" { availability_zone = data.tencentcloud_availability_zones.zone.zones[0].name image_id = data.tencentcloud_images.image.images[0].image_id instance_type = data.tencentcloud_instance_types.instanceType.instance_types[0].instance_type vpc_id = tencentcloud_vpc.vpc.id subnet_id = tencentcloud_subnet.subnet.id // allocate_public_ip = true system_disk_size = 50 system_disk_type = "CLOUD_PREMIUM" key_name = tencentcloud_key_pair.xuel-tf-key.id allocate_public_ip = true internet_max_bandwidth_out = 2

data_disks { data_disk_size = 50 data_disk_type = "CLOUD_PREMIUM" } hostname = format("xuel-tf-ansible-manager-%d", count.index) instance_charge_type = "POSTPAID_BY_HOUR" count = local.count instance_name = format("xuel-tf-server-%d", count.index)

tags = { tagkey = "xuel-ansible-manager" } }

// ansible 服务器部署ansible resource "null_resource" "shell" { depends_on = [tencentcloud_instance.xuel_tf_ansible] count = local.count

triggers = { instance_ids = element(tencentcloud_instance.xuel_tf_ansible..id, count.index) } provisioner "remote-exec" { connection { host = element(tencentcloud_instance.xuel_tf_ansible..public_ip, count.index) type = "ssh" user = "root" private_key = file("${local.key_file}") }

inline = [
  // 安装nginx
  "echo index.html > /xueltf.txt",
  "yum -y install nginx",
  "echo tf-nginx > /usr/share/nginx/html/index.html",
  "systemctl start nginx",
  "systemctl status nginx",

  // 安装ansible
  "yum install epel-release -y",
  "yum install ansible -y",

  // 配置ansible目录
  format("[ ! -d %s ] && mkdir -pv %s || echo 'dir create ok'", local.ansible_dir,local.ansible_dir)

]

} }

// 拷贝ansible文件至ansible 管理节点 resource "null_resource" "file_copy" { depends_on = [null_resource.shell]

connection { host = tencentcloud_instance.xuel_tf_ansible.0.public_ip user = local.username type = "ssh" private_key = file("${local.key_file}") }

// ansible 配置文件 provisioner "file" { source = local.local_ansible_config destination = format("${local.ansible_dir}/%s", local.local_ansible_config) } }

output "summary" { value = { // image = {for k,v in data.tencentcloud_images.image:k=>v} // instanceType = {for k,v in data.tencentcloud_instance_types // .instanceType:k=>v} // zone = {for k,v in data.tencentcloud_availability_zones.zone: k=>v} instance = { for k, v in tencentcloud_instance.xuel_tf_ansible : k => v }

ip = tencentcloud_instance.xuel_tf_ansible.0.public_ip

} } ```

3.2 编排node节点

  • 利用本地密钥编排多台node节点
  • 从ansible_manager 的 terraform_remote_state 获取ansible manger的ip地址信息，用于传输node节点的IP.txt 列表文件至ansible
  • 将本地编排node节点的ansible playbook脚本上次至目标服务器。
  • 将创建node节点的私钥传输到ansible，供后续ansible免密登录node节点执行ansible playbook

```shell locals { cidr_block_vpc_name = "10.0.0.0/16" cidr_block_subnet_name = "10.0.1.0/24" count = 3 local_ip_file = "ip.txt" ansible_dir = "/opt/ansibleworkspace/"

// 本地ansible play_bookfile = "instance.yaml" local_ansible_playbook_dir = "ansible_playbook" key_file = "id_rsa"

}

// 获取ansible_manager 的公网IP data "terraform_remote_state" "ansible_manager_ip" { backend = "cos" config = { region = "ap-beijing" bucket = "tfproject-1253329830" prefix = "ansible_manager/default" } }

data "tencentcloud_images" "image" { image_type = ["PUBLIC_IMAGE"] os_name = "centos 7.5" }

data "tencentcloud_instance_types" "instanceType" { cpu_core_count = 1 memory_size = 1 filter { name = "instance-family" values = ["S3"] } }

data "tencentcloud_availability_zones" "zone" {}

resource "tencentcloud_vpc" "vpc" { cidr_block = local.cidr_block_vpc_name name = "xuel_tf_vpc" }

resource "tencentcloud_subnet" "subnet" { availability_zone = data.tencentcloud_availability_zones.zone.zones.0.name cidr_block = local.cidr_block_subnet_name name = "xuel_tf_subnet" vpc_id = tencentcloud_vpc.vpc.id }

// 定义公钥 resource "tencentcloud_key_pair" "xuel-tf-key" { key_name = "xuel_node_key" public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCcJxzfjqQDikRRShExu22kvNOfMKeWrn++s5nQMTm+9SNsQISEk+P15JhFVachgjumupMcOIrfJQAAQcnzNmxoRCTCJQfmegGpDZVpE1cyHUhGMA5kwu67PmK1Snm8hkg2Xzlduhr1xysL2mRn3+6o5tsFXhGrYOcSSXnf5SpTPgMjqo339ksH0iv8kvu3NaZRueygLYaVEMjixJvsnUisL3uY8LQ+4cm2Zu5mdQamhWhN0kkSdlfbjPgzxexL4AglD9YDy4I9Q80vKzy33Ubwo17a2aNCF3uPpYvCKiV0H9z2XtMxisKDfsQQA01Q1vpccUIK6L48xSbers8hV2xxpSEWzEuoZg18eG2ikAencA6mhGjFWcp9A1dllY2rUhcEdrjcjXji1SGabjYLsv23ki6EMGjM/AK+fq+vj3pIPUMpscX3xVDGmz/zusq6v1KfOtQw7B/Dg8c2cxKUlEWZqqC3A7rt3JO/RVEbeqSe5mlRm2yngINVemmhkcfZNs= [email protected]" }

resource "tencentcloud_instance" "xuel_tf_ansible_node" { availability_zone = data.tencentcloud_availability_zones.zone.zones[0].name image_id = data.tencentcloud_images.image.images[0].image_id instance_type = data.tencentcloud_instance_types.instanceType.instance_types[0].instance_type vpc_id = tencentcloud_vpc.vpc.id subnet_id = tencentcloud_subnet.subnet.id // allocate_public_ip = true system_disk_size = 50 system_disk_type = "CLOUD_PREMIUM" key_name = tencentcloud_key_pair.xuel-tf-key.id allocate_public_ip = true internet_max_bandwidth_out = 2

data_disks { data_disk_size = 50 data_disk_type = "CLOUD_PREMIUM" } hostname = format("xuel-tf-server-%d", count.index) instance_charge_type = "POSTPAID_BY_HOUR" count = local.count instance_name = format("xuel-tf-server-%d", count.index)

tags = { name = "xuel_tf_ansibles" } }

// 生成IP地址文件 resource "local_file" "ip_local_file" { filename = local.local_ip_file content = join("\n", tencentcloud_instance.xuel_tf_ansible_node.*.public_ip) }

// 第一步，将生产的公网IP地址发送给远程ansible服务器 resource "null_resource" "copy_file" { depends_on = [tencentcloud_instance.xuel_tf_ansible_node]

connection { type = "ssh" host = data.terraform_remote_state.ansible_manager_ip.outputs.summary.ip user = var.ansible_user private_key = file("/Users/xuel/.ssh/id_rsa") }

// 生成的目标服务器的ip地址 provisioner "file" { source = local.local_ip_file destination = format("${local.ansible_dir}/%s", local.local_ip_file) }

// 到目标服务器执行的ansible playbook provisioner "file" { source = format("%s/%s", local.local_ansible_playbook_dir,local.play_bookfile ) destination = format("${local.ansible_dir}/%s", local.play_bookfile) }

// 到目标服务器的密钥 provisioner "file" { source = "/Users/xuel/.ssh/id_rsa" destination = "${local.ansible_dir}/${local.key_file}" }

// 目标服务器进行授权 provisioner "remote-exec" { inline = [ "chmod 600 ${local.ansible_dir}/${local.key_file}" ] }

}

// 第二步，登录ansible 控制节点执行ansible playbook resource "null_resource" "exec_ansible_playbook" { depends_on = [null_resource.copy_file]

provisioner "remote-exec" {

connection {
  type = "ssh"
  host = data.terraform_remote_state.ansible_manager_ip.outputs.summary.ip
  user = var.ansible_user
  private_key = file("/Users/xuel/.ssh/id_rsa")
}

inline = [
  "cd ${local.ansible_dir}",
  "ansible-playbook ${local.play_bookfile}"
]

} } ```

在对node节点编排出来后，使用以下ansible playbook进行对服务器执行配置管理，主要内容为安装http，git并启动http服务，对数据盘/dev/vdb进行格式化挂在，并在其写入html文件

```yaml - name: integration of terraform and ansible hosts: all

tasks: - name: installing httpd package: name: httpd state: present - name: installing php package: name: php state: present - name: starting httpd service service: name: httpd state: started - name: installing git package: name: git state: present - name: formatting storage filesystem: fstype: ext4 dev : /dev/vdb - name: making folder file: path: /var/www/html/web state: directory

- name: mounting storage
  mount:
    fstype: ext4
    src: /dev/vdb
    path: /var/www/html
    state: mounted

- name: create html file
  ansible.builtin.shell:
    cmd: echo "xuel tf ansible page" > index.html
    chdir: /var/www/html

```

四 测试

本实战将backend存储在腾讯云cos中，workspace使用default

4.1 编排ansible manger节点

进行编排ansible manger节点

查看存储在cos中的state文件

登录编排出来的ansible manger查看信息

4.2 编排node节点

再次编排三台node节点。

查看存储在cos中的state状态文件

登录任意服务器进行查看情况，http已经启动，并且磁盘已经格式化并挂载。

查看web页面

其他

  • 在写入多个ip列表时

content = join("\n", tencentcloud_instance.xuel_tf_ansible_node.*.public_ip)

  • ansible 的配置文件中定义了inventory 和 private_key_file 的路径

shell [defaults] inventory = /opt/ansibleworkspace/ip.txt host_key_checking= False private_key_file=/opt/ansibleworkspace/id_rsa remote_user=root [privilege_escalation] become=true become_method=sudo become_user=root become_ask_pass=false

后期可以结合将编排出的主机进行分组，并配合git流程实现基于Gitlab 的DevOPS流程。

参考链接

  • http://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource

其他

「」
「」