Posted Updated 13 minutes read (About 2005 words)

CoreDNS和NodeLocalDNS的域名解析

在 K8s 中，DNS 的解析主要用这两个工具:

  1. CoreDNS: 主要负责集群内部域名解析
  2. NodeLocalDNS: 提供 DNS 缓存

首先看一下集群中节点的 /etc/resolv.conf 配置文件
node-resolv-conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 指定了搜索域，当使用域名解析主机名时，如果主机名没有完全限定，系统会依次尝试在指定的搜索域中追加搜索后缀进行解析
# 例如解析 test，则会尝试解析 test.default.svc.cluster.local 和 test.svc.cluster.local
search default.svc.cluster.local svc.cluster.local

# nodelocaldns 服务器的地址，如果集群中没有 nodelocaldns，那么这个地址就会替换成 CoreDNS 的 SVC ClusterIP
nameserver 169.254.25.10

# 其他 DNS 服务器地址
nameserver 192.168.0.5
nameserver 223.5.5.5

# 解析的域名最多包涵 2 个点，最多超时 2s，最多重试 2 次
# 假设解析a，会尝试追加 default.svc.cluster.local 和 svc.cluster.local
# 假设解析a.b，则不会，除非 ndots > 2
options ndots:2 timeout:2 attempts:2

在 K8s 中，workload 的 dnsPolicy 有四种类型:

  1. ClusterFirst: 与配置的集群域后缀不匹配的任何 DNS 查询（例如 “www.kubernetes.io"） 都会由 DNS 服务器转发到上游名称服务器
  2. Default: 从运行所在的节点继承名称解析配置
  3. ClusterFirstWithHostNet: 对于以 hostNetwork 方式运行的 Pod，应将其 DNS 策略显式设置为 ClusterFirstWithHostNet。否则，以 hostNetwork 方式和 ClusterFirst 策略运行的 Pod 将会做出回退至 Default 策略的行为
  4. None: 会使用 dnsConfig 提供的 DNS 配置

CoreDNS

CoreDNS 主要是复杂集群内部域名的解析，保证 Pod 与 Pod 只见可以通过 Service Name 进行通信。当然也可以为它添加配置，使其能够解析静态的一些 Host 👇

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
kind: ConfigMap
apiVersion: v1
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors {
        }
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
        }
        prometheus :9153
        forward . /etc/resolv.conf {
          prefer_udp
          max_concurrent 1000
        }
        hosts /etc/coredns/Hosts {
          ttl 60
          reload 1m
          fallthrough
        }
        cache 30
        loop
        reload
        loadbalance
    }
  Hosts: |+
    192.168.0.10 cqm.com

创建一个 dnsPolicy 为 ClusterFirst 的 busybox
busybox

通过这个 busybox 解析集群内部域名，通过 tcpdump 获取 53 端口的包
通过coredns解析集群内部域名

可以看到会根据 search 对 CoreDNS 发起多次请求

Pod -> CoreDNS -> Pod

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[root@controller-node-1 ~]# tcpdump -i any host 10.233.0.3 and 10.233.74.83 and port 53 -nnvvv
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
22:52:20.447820 IP (tos 0x0, ttl 64, id 37856, offset 0, flags [DF], proto UDP (17), length 82)
    10.233.74.83.58663 > 10.233.0.3.53: [bad udp cksum 0x6077 -> 0x753d!] 33482+ A? kubernetes.default.svc.cluster.local. (54)
22:52:20.447950 IP (tos 0x0, ttl 64, id 37857, offset 0, flags [DF], proto UDP (17), length 74)
    10.233.74.83.58663 > 10.233.0.3.53: [bad udp cksum 0x606f -> 0xdd2e!] 53027+ A? kubernetes.svc.cluster.local. (46)
22:52:20.447978 IP (tos 0x0, ttl 64, id 37858, offset 0, flags [DF], proto UDP (17), length 70)
    10.233.74.83.58663 > 10.233.0.3.53: [bad udp cksum 0x606b -> 0x0aa3!] 30769+ A? kubernetes.cluster.local. (42)
22:52:20.447999 IP (tos 0x0, ttl 64, id 37859, offset 0, flags [DF], proto UDP (17), length 82)
    10.233.74.83.58663 > 10.233.0.3.53: [bad udp cksum 0x6077 -> 0x2dde!] 51726+ AAAA? kubernetes.default.svc.cluster.local. (54)
22:52:20.448017 IP (tos 0x0, ttl 64, id 37860, offset 0, flags [DF], proto UDP (17), length 74)
    10.233.74.83.58663 > 10.233.0.3.53: [bad udp cksum 0x606f -> 0xba24!] 61970+ AAAA? kubernetes.svc.cluster.local. (46)
22:52:20.448038 IP (tos 0x0, ttl 64, id 37861, offset 0, flags [DF], proto UDP (17), length 70)
    10.233.74.83.58663 > 10.233.0.3.53: [bad udp cksum 0x606b -> 0x7758!] 2913+ AAAA? kubernetes.cluster.local. (42)
22:52:20.448407 IP (tos 0x0, ttl 63, id 32091, offset 0, flags [DF], proto UDP (17), length 163)
    10.233.0.3.53 > 10.233.74.83.58663: [bad udp cksum 0x60c8 -> 0xd926!] 2913 NXDomain*- q: AAAA? kubernetes.cluster.local. 0/1/0 ns: cluster.local. [5s] SOA ns.dns.cluster.local. hostmaster.cluster.local. 1711809447 7200 1800 86400 5 (135)
22:52:20.448557 IP (tos 0x0, ttl 63, id 32092, offset 0, flags [DF], proto UDP (17), length 175)
    10.233.0.3.53 > 10.233.74.83.58663: [bad udp cksum 0x60d4 -> 0x8faf!] 51726*- q: AAAA? kubernetes.default.svc.cluster.local. 0/1/0 ns: cluster.local. [5s] SOA ns.dns.cluster.local. hostmaster.cluster.local. 1711809447 7200 1800 86400 5 (147)
22:52:20.448654 IP (tos 0x0, ttl 63, id 32093, offset 0, flags [DF], proto UDP (17), length 163)
    10.233.0.3.53 > 10.233.74.83.58663: [bad udp cksum 0x60c8 -> 0x6c71!] 30769 NXDomain*- q: A? kubernetes.cluster.local. 0/1/0 ns: cluster.local. [5s] SOA ns.dns.cluster.local. hostmaster.cluster.local. 1711809447 7200 1800 86400 5 (135)
22:52:20.448699 IP (tos 0x0, ttl 63, id 32094, offset 0, flags [DF], proto UDP (17), length 167)
    10.233.0.3.53 > 10.233.74.83.58663: [bad udp cksum 0x60cc -> 0x1bf3!] 61970 NXDomain*- q: AAAA? kubernetes.svc.cluster.local. 0/1/0 ns: cluster.local. [5s] SOA ns.dns.cluster.local. hostmaster.cluster.local. 1711809447 7200 1800 86400 5 (139)
22:52:20.448761 IP (tos 0x0, ttl 63, id 32095, offset 0, flags [DF], proto UDP (17), length 134)
    10.233.0.3.53 > 10.233.74.83.58663: [bad udp cksum 0x60ab -> 0x24fc!] 33482*- q: A? kubernetes.default.svc.cluster.local. 1/0/0 kubernetes.default.svc.cluster.local. [5s] A 10.233.0.1 (106)
22:52:20.448811 IP (tos 0x0, ttl 63, id 32096, offset 0, flags [DF], proto UDP (17), length 167)
    10.233.0.3.53 > 10.233.74.83.58663: [bad udp cksum 0x60cc -> 0x3efd!] 53027 NXDomain*- q: A? kubernetes.svc.cluster.local. 0/1/0 ns: cluster.local. [5s] SOA ns.dns.cluster.local. hostmaster.cluster.local. 1711809447 7200 1800 86400 5 (139)

通过这个 busybox 解析集群公网域名
通过coredns解析集群公网域名

可以看到对 www.baidu.com 域名的解析返回结果，是通过 192.168.0.5 获取到的

Pod -> CoreDNS -> 192.168.0.5 -> CoreDNS -> Pod

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[root@controller-node-1 ~]#  tcpdump -i any host 10.233.74.83 and 10.233.0.3 or 192.168.0.5 and port 53 -nnvvv
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
22:58:29.405332 IP (tos 0x0, ttl 64, id 48447, offset 0, flags [DF], proto UDP (17), length 59)
    10.233.74.83.38430 > 10.233.0.3.53: [bad udp cksum 0x6060 -> 0xdcc6!] 28765+ A? www.baidu.com. (31)
22:58:29.405423 IP (tos 0x0, ttl 64, id 48448, offset 0, flags [DF], proto UDP (17), length 59)
    10.233.74.83.38430 > 10.233.0.3.53: [bad udp cksum 0x6060 -> 0xcdcd!] 25686+ AAAA? www.baidu.com. (31)
22:58:29.405881 IP (tos 0x0, ttl 64, id 42712, offset 0, flags [DF], proto UDP (17), length 59)
    10.233.74.66.56555 > 192.168.0.5.53: [bad udp cksum 0x1611 -> 0x2f90!] 1558+ AAAA? www.baidu.com. (31)
22:58:29.405950 IP (tos 0x0, ttl 63, id 42712, offset 0, flags [DF], proto UDP (17), length 59)
    192.168.159.41.56555 > 192.168.0.5.53: [bad udp cksum 0x20b8 -> 0x24e9!] 1558+ AAAA? www.baidu.com. (31)
22:58:29.406110 IP (tos 0x0, ttl 64, id 42713, offset 0, flags [DF], proto UDP (17), length 59)
    10.233.74.66.58403 > 192.168.0.5.53: [bad udp cksum 0x1611 -> 0xf518!] 21589+ A? www.baidu.com. (31)
22:58:29.406191 IP (tos 0x0, ttl 63, id 42713, offset 0, flags [DF], proto UDP (17), length 59)
    192.168.159.41.58403 > 192.168.0.5.53: [bad udp cksum 0x20b8 -> 0xea71!] 21589+ A? www.baidu.com. (31)
22:58:29.410592 IP (tos 0x0, ttl 63, id 39983, offset 0, flags [DF], proto UDP (17), length 118)
    192.168.0.5.53 > 192.168.159.41.58403: [udp sum ok] 21589 q: A? www.baidu.com. 3/0/0 www.baidu.com. [2m9s] CNAME www.a.shifen.com., www.a.shifen.com. [2m9s] A 183.2.172.185, www.a.shifen.com. [2m9s] A 183.2.172.42 (90)
22:58:29.410637 IP (tos 0x0, ttl 62, id 39983, offset 0, flags [DF], proto UDP (17), length 118)
    192.168.0.5.53 > 10.233.74.66.58403: [udp sum ok] 21589 q: A? www.baidu.com. 3/0/0 www.baidu.com. [2m9s] CNAME www.a.shifen.com., www.a.shifen.com. [2m9s] A 183.2.172.185, www.a.shifen.com. [2m9s] A 183.2.172.42 (90)
22:58:29.410972 IP (tos 0x0, ttl 63, id 64831, offset 0, flags [DF], proto UDP (17), length 166)
    10.233.0.3.53 > 10.233.74.83.38430: [bad udp cksum 0x60cb -> 0xe43e!] 28765 q: A? www.baidu.com. 3/0/0 www.baidu.com. [30s] CNAME www.a.shifen.com., www.a.shifen.com. [30s] A 183.2.172.42, www.a.shifen.com. [30s] A 183.2.172.185 (138)
22:58:29.413159 IP (tos 0x0, ttl 63, id 39984, offset 0, flags [DF], proto UDP (17), length 142)
    192.168.0.5.53 > 192.168.159.41.56555: [udp sum ok] 1558 q: AAAA? www.baidu.com. 3/0/0 www.baidu.com. [3m29s] CNAME www.a.shifen.com., www.a.shifen.com. [3m29s] AAAA 240e:ff:e020:9ae:0:ff:b014:8e8b, www.a.shifen.com. [3m29s] AAAA 240e:ff:e020:966:0:ff:b042:f296 (114)
22:58:29.413203 IP (tos 0x0, ttl 62, id 39984, offset 0, flags [DF], proto UDP (17), length 142)
    192.168.0.5.53 > 10.233.74.66.56555: [udp sum ok] 1558 q: AAAA? www.baidu.com. 3/0/0 www.baidu.com. [3m29s] CNAME www.a.shifen.com., www.a.shifen.com. [3m29s] AAAA 240e:ff:e020:9ae:0:ff:b014:8e8b, www.a.shifen.com. [3m29s] AAAA 240e:ff:e020:966:0:ff:b042:f296 (114)
22:58:29.413922 IP (tos 0x0, ttl 63, id 64833, offset 0, flags [DF], proto UDP (17), length 190)
    10.233.0.3.53 > 10.233.74.83.38430: [bad udp cksum 0x60e3 -> 0x9ac9!] 25686 q: AAAA? www.baidu.com. 3/0/0 www.baidu.com. [30s] CNAME www.a.shifen.com., www.a.shifen.com. [30s] AAAA 240e:ff:e020:966:0:ff:b042:f296, www.a.shifen.com. [30s] AAAA 240e:ff:e020:9ae:0:ff:b014:8e8b (162)

NodeLocalDNS

来自官方文档: NodeLocal DNSCache 通过在集群节点上作为 DaemonSet 运行 DNS 缓存代理来提高集群 DNS 性能。

NodeLocalDNS Pod 中的 /etc/resolv.conf 与宿主机是相同的
nodelocaldns的resolv配置

节点上也会有对应的网卡，这个地址取决于启动参数
nodelocaldns网卡

在集群有 NodeLocalDNS 的情况下，在对集群内部域名进行解析，可以看到 DNS 请求会先经过 NodeLocalDNS，然后直接返回请求

Pod -> NodeLocalDNS -> Pod

1
2
3
4
5
6
7
8
9
10
[root@controller-node-1 ~]# tcpdump -i any host 10.233.74.83 and 169.254.25.10 or 10.233.0.3 and port 53 -nnvvv
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
23:19:46.273718 IP (tos 0x0, ttl 64, id 61227, offset 0, flags [DF], proto UDP (17), length 82)
    10.233.74.83.36711 > 169.254.25.10.53: [bad udp cksum 0x1894 -> 0x3f9b!] 22032+ A? kubernetes.default.svc.cluster.local. (54)
23:19:46.273793 IP (tos 0x0, ttl 64, id 61228, offset 0, flags [DF], proto UDP (17), length 82)
    10.233.74.83.36711 > 169.254.25.10.53: [bad udp cksum 0x1894 -> 0xb26b!] 58148+ AAAA? kubernetes.default.svc.cluster.local. (54)
23:19:46.275678 IP (tos 0x0, ttl 64, id 60479, offset 0, flags [DF], proto UDP (17), length 134)
    169.254.25.10.53 > 10.233.74.83.36711: [bad udp cksum 0x18c8 -> 0xef59!] 22032*- q: A? kubernetes.default.svc.cluster.local. 1/0/0 kubernetes.default.svc.cluster.local. [5s] A 10.233.0.1 (106)
23:19:46.275946 IP (tos 0x0, ttl 64, id 60480, offset 0, flags [DF], proto UDP (17), length 175)
    169.254.25.10.53 > 10.233.74.83.36711: [bad udp cksum 0x18f1 -> 0x5d58!] 58148*- q: AAAA? kubernetes.default.svc.cluster.local. 0/1/0 ns: cluster.local. [5s] SOA ns.dns.cluster.local. hostmaster.cluster.local. 1711802462 7200 1800 86400 5 (147)

然后在对集群外部域名进行解析

Pod -> NodeLocalDNS -> CoreDNS -> 192.168.0.5 -> CoreDNS -> NodeLocalDNS -> Pod

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
23:27:25.569362 IP (tos 0x0, ttl 64, id 36019, offset 0, flags [DF], proto UDP (17), length 60)
    10.233.74.83.42419 > 169.254.25.10.53: [bad udp cksum 0x187e -> 0x94cd!] 7676+ A? www.google.com. (32)
23:27:25.569423 IP (tos 0x0, ttl 64, id 36020, offset 0, flags [DF], proto UDP (17), length 60)
    10.233.74.83.42419 > 169.254.25.10.53: [bad udp cksum 0x187e -> 0xb5c9!] 64740+ AAAA? www.google.com. (32)
23:27:25.569823 IP (tos 0x0, ttl 64, id 65347, offset 0, flags [DF], proto UDP (17), length 71)
    192.168.159.11.57457 > 192.168.0.5.53: [bad udp cksum 0x20a6 -> 0x7971!] 52418+ [1au] AAAA? www.google.com. ar: . OPT UDPsize=2048 DO (43)
23:27:25.569841 IP (tos 0x0, ttl 64, id 65348, offset 0, flags [DF], proto UDP (17), length 71)
    192.168.159.11.33304 > 192.168.0.5.53: [bad udp cksum 0x20a6 -> 0xa430!] 120+ [1au] A? www.google.com. ar: . OPT UDPsize=2048 DO (43)
23:27:25.571423 IP (tos 0x0, ttl 63, id 18081, offset 0, flags [DF], proto UDP (17), length 152)
    192.168.0.5.53 > 192.168.159.11.57457: [udp sum ok] 52418 q: AAAA? www.google.com. 0/1/1 ns: www.google.com. [10s] SOA fake-for-negative-caching.adguard.com. hostmaster.www.google.com. 100500 1800 60 604800 86400 ar: . OPT UDPsize=2048 DO (124)
23:27:25.571763 IP (tos 0x0, ttl 64, id 40563, offset 0, flags [DF], proto UDP (17), length 172)
    169.254.25.10.53 > 10.233.74.83.42419: [bad udp cksum 0x18ee -> 0x3576!] 64740 q: AAAA? www.google.com. 0/1/0 ns: www.google.com. [10s] SOA fake-for-negative-caching.adguard.com. hostmaster.www.google.com. 100500 1800 60 604800 86400 (144)
23:27:25.573701 IP (tos 0x0, ttl 63, id 18082, offset 0, flags [DF], proto UDP (17), length 87)
    192.168.0.5.53 > 192.168.159.11.33304: [udp sum ok] 120 q: A? www.google.com. 1/0/1 www.google.com. [1m30s] A 216.58.203.68 ar: . OPT UDPsize=2048 DO (59)
23:27:25.573927 IP (tos 0x0, ttl 64, id 40564, offset 0, flags [DF], proto UDP (17), length 90)
    169.254.25.10.53 > 10.233.74.83.42419: [bad udp cksum 0x189c -> 0xe2c7!] 7676 q: A? www.google.com. 1/0/0 www.google.com. [30s] A 216.58.203.68 (62)

CoreDNS和NodeLocalDNS的域名解析

https://warnerchen.github.io/2024/03/30/CoreDNS和NodeLocalDNS的域名解析/

Author

Warner Chen

Posted on

2024-03-30

Updated on

2024-03-31

Licensed under

#
You need to set install_url to use ShareThis. Please set it in _config.yml.

Like this article? Support the author with

You forgot to set the business or currency_code for Paypal. Please set it in _config.yml.

Comments

You forgot to set the shortname for Disqus. Please set it in _config.yml.
Follow

Links

Recents

Archives

Tags

ansible1
calico1
cisco1
container2
containerd1
docker1
elk1
etcd3
istio2
jenkins1
k8s10
kafka1
linux3
mysql1
nexus1
prometheus1
redis1
suse10
web1
zookeeper1
面试1
首页1

Subscribe for updates

You need to set client_id and slot_id to show this AD unit. Please set it in _config.yml.

follow.it

×