技术

Appearance

流的定义 ​

流是一系列通过网络中某一观察点的具有相同属性的数据包，属性包括

  • 端点：流的起点和终点
  • 方向：单向或双向
  • 时间粒度：数据包的发送起始时间和终止时间
  • 协议层次：包括网络各层协议

NetFlow/IPFIX协议介绍与关系 ​

  • Netflow: 于1996年由思科公司发明，于同年5月注册为美国专利。首先用于网络设备对数据交换进行加速，并同步实现对高速转发的IP数据流进行测量和统计，经过多年的技术演进，NetFlow原来用于数据交换加速的功能已经逐步由网络设备中的专用ASIC 芯片实现，而对IP数据流进行测量和统计的功能也已更加成熟，形成了一种专用流交换技术，NetFlow已集成到大多数厂商的路由器和交换机内成为流量监测的事实标准，广泛应用于网络管理
  • IPFIX：全称IP Flow Information Export，即IP流信息输出，在 2003 年，Netflow V9 被 IETF 组织从 5 个候选方案中确定为 IPFIX标准，作为IP网络中的流信息测量的标准协议，是网络流量监测的国际标准，是一种针对数据流特征分析、基于模板的格式输出的协议，因此具有很强的可扩展性

协议缺点 ​

  • netflow:
    • 消耗路由器的CPU和存储资源，对设备的转发性能有影响
    • 流记录的数据量仍然很大，非独立设备，处理能力不足，因此，一般采用100～1000:1采样比，监测粒度较粗，损失流量 的细节信息
    • 设备本身提供的数据内容有局限性，策略和定制能力相对较差。NetFlow功能集成在设备内，软件实现性能差、硬件实现灵活性差，对于大量已部署运行的路由器升级板卡费用高
    • NetFlow业务和应用识别主要依赖于TCP/UDP端口号，无法识别日新月异的业务

协议系统组成 ​

  • Exporter/探测器：用于监听网络数据
  • Collector/采集器：用于收集从 Exporter 输出的网络数据
  • Analysis/分析报告系统：用于分析从 Collector 收集到的网络数据，并产生报告

netflow数据字段 ​

数据流

  • 源IP
  • 目的IP
  • 协议
  • 端口

流量统计数据

  • 数据流时戳
  • 源 IP 地址
  • 目的 IP 地址
  • 源端口号
  • 目的端口号
  • 输入接口号和输出接口号
  • 下一跳 IP 地址
  • 信息流中的总字节数
  • 信息流中的数据包数量
  • 信息流中的第一个和最后一个数据包时戳 源 AS 和目的 AS
  • 前置掩码
  • 数据包序号

IPFIX数据字段 ​

为了较完整的输出数据，IPFIX 缺省使用网络设备的七个关键域来表示每股网络流量，如果不同的 IP 报文中所有的七个关键域都匹配，那么这些 IP 报文都将被视为属于同一股流量

通过记录网络中这些流量的特征，如流量持续时间、流量中报文平均长度等， 我们可以了解到当前网络的应用情况，并根据这些信息对网络进行优化，安全检测，流量计费

  • 源 IP 地址
  • 目的IP 地址
  • TCP/UDP 源端口
  • TCP/UDP 目的端口
  • 三层协议类型
  • 服务类型Type-of-service字节
  • 输入逻辑接口

协议收集器collector ​

依赖开源项目goflow2(golang项目)

https://github.com/netsampler/goflow2
https://github.com/netsampler/goflow2

构建

shell
$ git clone https://github.com/netsampler/goflow2.git
$ cd goflow2
$ make build
# 如果需要静态链接库，可以执行如下
$ CGO_ENABLED=0 make build
$ git clone https://github.com/netsampler/goflow2.git
$ cd goflow2
$ make build
# 如果需要静态链接库，可以执行如下
$ CGO_ENABLED=0 make build

会产生一个dist文件夹，里面包含一个可执行文件dist/goflow2-版本-git-commit-架构

把可执行文件重命名为goflow2

执行监听，可以看到该程序同时支持netflow/sflow协议采集，监听udp2055端口收集netflow协议数据包，保持这个终端不关闭，接下去开始执行协议探测程序

shell
$ ./dist/goflow2 
INFO[0000] starting GoFlow2                             
INFO[0000] starting collection                           count=1 hostname= port=6343 scheme=sflow
INFO[0000] starting collection                           count=1 hostname= port=2055 scheme=netflow
$ ./dist/goflow2 
INFO[0000] starting GoFlow2                             
INFO[0000] starting collection                           count=1 hostname= port=6343 scheme=sflow
INFO[0000] starting collection                           count=1 hostname= port=2055 scheme=netflow

协议探测exporter ​

依赖开源项目softflowd（c语言项目）作为流量探测器，可以探测linux主机物理网口的网卡流量信息，并且按照netflow/ipfix协议定义封装流量包发送到指定的采集器

https://github.com/irino/softflowd
https://github.com/irino/softflowd

构建，如下操作会生成可执行文件softflowd

shell
$ git clone https://github.com/irino/softflowd.git
$ cd softflowd
$ autoreconf -if
$ ./configure
$ make
$ git clone https://github.com/irino/softflowd.git
$ cd softflowd
$ autoreconf -if
$ ./configure
$ make

如果遇到报错如下需要安装automake

$ autoreconf -if
Can't exec "aclocal": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 326.
autoreconf: failed to run aclocal: No such file or directory
$ autoreconf -if
Can't exec "aclocal": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 326.
autoreconf: failed to run aclocal: No such file or directory

解决方案

  • sudo apt/yum/dnf install automake

  • 从源码构建，访问如下地址获取源码包

    https://www.gnu.org/software/automake/
    https://www.gnu.org/software/automake/
    $ cd automake***
$ ./configure
$ make 
# make 报错执行下面命令再重新执行make
# $ sed -i 's/$(update_mans) automake-$(APIVERSION)/$(update_mans) automake-$(APIVERSION) --no-discard-stderr/g' Makefile
$ make install
    $ cd automake***
$ ./configure
$ make 
# make 报错执行下面命令再重新执行make
# $ sed -i 's/$(update_mans) automake-$(APIVERSION)/$(update_mans) automake-$(APIVERSION) --no-discard-stderr/g' Makefile
$ make install

安装到/usr/bin(这一步可以省略)

shell
$ sudo make install
$ sudo make install

执行netflow采集

shell
$ sudo ./softflowd -v 5 -i enp3s0 -t maxlife=1 -n 127.0.0.1:2055 -P udp -D
# 或者
$ sudo ./softflowd -v 9 -i enp3s0 -t maxlife=1 -n 127.0.0.1:2055 -P udp -D
$ sudo ./softflowd -v 5 -i enp3s0 -t maxlife=1 -n 127.0.0.1:2055 -P udp -D
# 或者
$ sudo ./softflowd -v 9 -i enp3s0 -t maxlife=1 -n 127.0.0.1:2055 -P udp -D

执行ipfix采集

shell
$ sudo ./softflowd -v 10 -i enp3s0 -t maxlife=1 -n 127.0.0.1:2055 -P udp -D
$ sudo ./softflowd -v 10 -i enp3s0 -t maxlife=1 -n 127.0.0.1:2055 -P udp -D

参数解释

  • -v：表示发送netflow协议版本号，一般是5/9版本，依据./softflowd --help输出信息，如下表示，当版本是10的时候，会探测并封装为ipfix包

      -v 1|5|9|10|psamp       NetFlow export packet version
                          10 means IPFIX and psamp means PSAMP (packet sampling)
      -v 1|5|9|10|psamp       NetFlow export packet version
                          10 means IPFIX and psamp means PSAMP (packet sampling)

  • -i enp3s0: 表示探测当前主机网口enp3s0的网卡流量，主机网卡可以通过ip a命令获取

  • -t maxlife=3: 表示每3秒发送一次netflow流包，目前softflowd 1.1.0版本help命令输出参数是 -t timeout=time Specify named timeout，后期如果maxlife无效，可以尝试修改为-t timeout=1

  • -n 127.0.0.1:2055：表示把探测到的流量数据封装并发送给目的地址为127.0.0.1:2055的collector

  • -P udp: 指定发送给exporter的传输层协议类型，一般是udp

  • -D: debug模式，保持前端运行并输出日志信息

exporter程序打印数据 ​

当softflowd的-v参数变化时，goflow2 exporter监听程序打印输出的数据会动态输出不同的类型

-v 5

json
{
  "type": "NETFLOW_V5",
  "time_received_ns": 0,
  "sequence_num": 4,
  "sampling_rate": 0,
  "sampler_address": "127.0.0.1",
  "time_flow_start_ns": 1692162613180684000,
  "time_flow_end_ns": 1692166704180684000,
  "bytes": 642,
  "packets": 6,
  "src_addr": "10.30.2.174",
  "dst_addr": "239.192.152.143",
  "etype": "IPv4",
  "proto": "UDP",
  "src_port": 7000,
  "dst_port": 7000,
  "in_if": 0,
  "out_if": 0,
  "src_mac": "00:00:00:00:00:00",
  "dst_mac": "00:00:00:00:00:00",
  "src_vlan": 0,
  "dst_vlan": 0,
  "vlan_id": 0,
  "ip_tos": 0,
  "forwarding_status": 0,
  "ip_ttl": 0,
  "tcp_flags": 0,
  "icmp_type": 0,
  "icmp_code": 0,
  "ipv6_flow_label": 0,
  "fragment_id": 0,
  "fragment_offset": 0,
  "src_as": 0,
  "dst_as": 0,
  "next_hop": "0.0.0.0",
  "next_hop_as": 0,
  "src_net": "0.0.0.0/0",
  "dst_net": "0.0.0.0/0",
  "bgp_next_hop": "",
  "bgp_communities": [],
  "as_path": [],
  "mpls_ttl": [],
  "mpls_label": [],
  "mpls_ip": [],
  "observation_domain_id": 0,
  "observation_point_id": 0
}
{
  "type": "NETFLOW_V5",
  "time_received_ns": 0,
  "sequence_num": 4,
  "sampling_rate": 0,
  "sampler_address": "127.0.0.1",
  "time_flow_start_ns": 1692162613180684000,
  "time_flow_end_ns": 1692166704180684000,
  "bytes": 642,
  "packets": 6,
  "src_addr": "10.30.2.174",
  "dst_addr": "239.192.152.143",
  "etype": "IPv4",
  "proto": "UDP",
  "src_port": 7000,
  "dst_port": 7000,
  "in_if": 0,
  "out_if": 0,
  "src_mac": "00:00:00:00:00:00",
  "dst_mac": "00:00:00:00:00:00",
  "src_vlan": 0,
  "dst_vlan": 0,
  "vlan_id": 0,
  "ip_tos": 0,
  "forwarding_status": 0,
  "ip_ttl": 0,
  "tcp_flags": 0,
  "icmp_type": 0,
  "icmp_code": 0,
  "ipv6_flow_label": 0,
  "fragment_id": 0,
  "fragment_offset": 0,
  "src_as": 0,
  "dst_as": 0,
  "next_hop": "0.0.0.0",
  "next_hop_as": 0,
  "src_net": "0.0.0.0/0",
  "dst_net": "0.0.0.0/0",
  "bgp_next_hop": "",
  "bgp_communities": [],
  "as_path": [],
  "mpls_ttl": [],
  "mpls_label": [],
  "mpls_ip": [],
  "observation_domain_id": 0,
  "observation_point_id": 0
}

-v 9

json
{
  "type": "NETFLOW_V9",
  "time_received_ns": 1692168027637104030,
  "sequence_num": 1,
  "sampling_rate": 0,
  "sampler_address": "127.0.0.1",
  "time_flow_start_ns": 1692162288000000000,
  "time_flow_end_ns": 1692167289000000000,
  "bytes": 276,
  "packets": 6,
  "src_addr": "10.30.7.161",
  "dst_addr": "224.0.0.18",
  "etype": "IPv4",
  "proto": "",
  "src_port": 0,
  "dst_port": 0,
  "in_if": 0,
  "out_if": 0,
  "src_mac": "00:00:00:00:00:00",
  "dst_mac": "00:00:00:00:00:00",
  "src_vlan": 0,
  "dst_vlan": 0,
  "vlan_id": 0,
  "ip_tos": 192,
  "forwarding_status": 0,
  "ip_ttl": 0,
  "tcp_flags": 0,
  "icmp_type": 0,
  "icmp_code": 0,
  "ipv6_flow_label": 0,
  "fragment_id": 0,
  "fragment_offset": 0,
  "src_as": 0,
  "dst_as": 0,
  "next_hop": "",
  "next_hop_as": 0,
  "src_net": "0.0.0.0/0",
  "dst_net": "0.0.0.0/0",
  "bgp_next_hop": "",
  "bgp_communities": [],
  "as_path": [],
  "mpls_ttl": [],
  "mpls_label": [],
  "mpls_ip": [],
  "observation_domain_id": 0,
  "observation_point_id": 0
}
{
  "type": "NETFLOW_V9",
  "time_received_ns": 1692168027637104030,
  "sequence_num": 1,
  "sampling_rate": 0,
  "sampler_address": "127.0.0.1",
  "time_flow_start_ns": 1692162288000000000,
  "time_flow_end_ns": 1692167289000000000,
  "bytes": 276,
  "packets": 6,
  "src_addr": "10.30.7.161",
  "dst_addr": "224.0.0.18",
  "etype": "IPv4",
  "proto": "",
  "src_port": 0,
  "dst_port": 0,
  "in_if": 0,
  "out_if": 0,
  "src_mac": "00:00:00:00:00:00",
  "dst_mac": "00:00:00:00:00:00",
  "src_vlan": 0,
  "dst_vlan": 0,
  "vlan_id": 0,
  "ip_tos": 192,
  "forwarding_status": 0,
  "ip_ttl": 0,
  "tcp_flags": 0,
  "icmp_type": 0,
  "icmp_code": 0,
  "ipv6_flow_label": 0,
  "fragment_id": 0,
  "fragment_offset": 0,
  "src_as": 0,
  "dst_as": 0,
  "next_hop": "",
  "next_hop_as": 0,
  "src_net": "0.0.0.0/0",
  "dst_net": "0.0.0.0/0",
  "bgp_next_hop": "",
  "bgp_communities": [],
  "as_path": [],
  "mpls_ttl": [],
  "mpls_label": [],
  "mpls_ip": [],
  "observation_domain_id": 0,
  "observation_point_id": 0
}

-v 10

json
{
  "type": "IPFIX",
  "time_received_ns": 1692168053184893251,
  "sequence_num": 25,
  "sampling_rate": 0,
  "sampler_address": "127.0.0.1",
  "time_flow_start_ns": 1692168053000000000,
  "time_flow_end_ns": 1692168053000000000,
  "bytes": 68,
  "packets": 1,
  "src_addr": "10.30.6.69",
  "dst_addr": "255.255.255.255",
  "etype": "IPv4",
  "proto": "UDP",
  "src_port": 40408,
  "dst_port": 60006,
  "in_if": 0,
  "out_if": 0,
  "src_mac": "00:00:00:00:00:00",
  "dst_mac": "00:00:00:00:00:00",
  "src_vlan": 0,
  "dst_vlan": 0,
  "vlan_id": 0,
  "ip_tos": 0,
  "forwarding_status": 0,
  "ip_ttl": 0,
  "tcp_flags": 0,
  "icmp_type": 0,
  "icmp_code": 0,
  "ipv6_flow_label": 0,
  "fragment_id": 0,
  "fragment_offset": 0,
  "src_as": 0,
  "dst_as": 0,
  "next_hop": "",
  "next_hop_as": 0,
  "src_net": "0.0.0.0/0",
  "dst_net": "0.0.0.0/0",
  "bgp_next_hop": "",
  "bgp_communities": [],
  "as_path": [],
  "mpls_ttl": [],
  "mpls_label": [],
  "mpls_ip": [],
  "observation_domain_id": 0,
  "observation_point_id": 0
}
{
  "type": "IPFIX",
  "time_received_ns": 1692168053184893251,
  "sequence_num": 25,
  "sampling_rate": 0,
  "sampler_address": "127.0.0.1",
  "time_flow_start_ns": 1692168053000000000,
  "time_flow_end_ns": 1692168053000000000,
  "bytes": 68,
  "packets": 1,
  "src_addr": "10.30.6.69",
  "dst_addr": "255.255.255.255",
  "etype": "IPv4",
  "proto": "UDP",
  "src_port": 40408,
  "dst_port": 60006,
  "in_if": 0,
  "out_if": 0,
  "src_mac": "00:00:00:00:00:00",
  "dst_mac": "00:00:00:00:00:00",
  "src_vlan": 0,
  "dst_vlan": 0,
  "vlan_id": 0,
  "ip_tos": 0,
  "forwarding_status": 0,
  "ip_ttl": 0,
  "tcp_flags": 0,
  "icmp_type": 0,
  "icmp_code": 0,
  "ipv6_flow_label": 0,
  "fragment_id": 0,
  "fragment_offset": 0,
  "src_as": 0,
  "dst_as": 0,
  "next_hop": "",
  "next_hop_as": 0,
  "src_net": "0.0.0.0/0",
  "dst_net": "0.0.0.0/0",
  "bgp_next_hop": "",
  "bgp_communities": [],
  "as_path": [],
  "mpls_ttl": [],
  "mpls_label": [],
  "mpls_ip": [],
  "observation_domain_id": 0,
  "observation_point_id": 0
}

阅读参考 ​

Linux主机安装NetFlow采集器并使用Graylog进行网络流量分析

Netflow/IPFIX 流量收集与分析

Last updated:

Released under the MIT License.

Copyright © 2023 gongzhengyang