技术

Appearance

协议介绍 ​

  • Sampled Flow的简称，由Inmon提出，是一种用于监控数据网络上交换机或者路由器流量转发状况，以设备端口为基本采样单元技术

  • 通过特定的采样技术获取网络设备上的流量转发统计并实时地通过sFlow数据报文发送到Collector以供分析

  • 报文采用UDP封装，缺省目的端口号为知名端口6343

  • 共有4种报文头格式，分别为Flow sample、Counter sample，其中v5版本新增Expanded Flow sample、Expanded Counter sample

协议优点 ​

  • 节省资源、降低成本：由于不需要建立流表，对网络设备的资源占用少，实现成本低
  • 采集器灵活、随需的部署：由于网络流的分析和统计工作由采集器完成，采集器可以灵活的配置网络流特征进行统计分析，实现灵活部署

组成 ​

  • sFlow Agent是客户端设备，一般是内嵌于网络转发设备中，比如交换机和路由器，负责收集设备上的流量转发情况
  • sFlow Collector是收集分析端设备，通常由专门服务器充当，监听UDP端口接收报文，进行解析以及数据分析展示

采样方式 ​

  • Flow采样：指定端口上按照特定的采样方向和采样比对报文进行采样分析，并将分析的结果通过sFlow报文发送到Collector设备的过程
  • Counter采样：设备周期性的获取接口上的流量统计，并将这些统计信息通过sFlow报文发送给Collector设备的过程

sFlow agent端配置与使用 ​

依赖项目host-sflow作为agent端，用于采集网口数据并发送给collector，可以采集当前主机设备网卡流量

支持系统包括linux/windows/freebsd/SunOS

https://github.com/sflow/host-sflow
https://github.com/sflow/host-sflow

从源码构建

shell
$ git clone https://github.com/sflow/host-sflow
$ cd host-sflow
$ make
$ git clone https://github.com/sflow/host-sflow
$ cd host-sflow
$ make

由于当前在linux下操作，所以生成的文件以及执行的make在src/Linux下面，多了一个src/Linux/hsflowd执行文件

安装到/usr/bin以及systemd注册(这一步可以选择执行)

shell
$ sudo make install
$ sudo make install

查看该文件链接情况，发现有一些动态依赖库

$ ldd src/Linux/hsflowd
        linux-vdso.so.1 (0x00007fff90b27000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f956e600000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f956e9b9000)
$ ldd src/Linux/hsflowd
        linux-vdso.so.1 (0x00007fff90b27000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f956e600000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f956e9b9000)

如果需要静态链接，可以找到src/Linux/Makefile，修改

makefile
hsflowd: $(OBJS_HSFLOWD) $(HEADERS)
	$(CC) $(CFLAGS) -o $@ $(OBJS_HSFLOWD) $(LIBS_HSFLOWD) $(LDFLAGS_HSFLOWD)
# 修改为 最后增加一个参数-static
hsflowd: $(OBJS_HSFLOWD) $(HEADERS)
	$(CC) $(CFLAGS) -o $@ $(OBJS_HSFLOWD) $(LIBS_HSFLOWD) $(LDFLAGS_HSFLOWD) -static
hsflowd: $(OBJS_HSFLOWD) $(HEADERS)
	$(CC) $(CFLAGS) -o $@ $(OBJS_HSFLOWD) $(LIBS_HSFLOWD) $(LDFLAGS_HSFLOWD)
# 修改为 最后增加一个参数-static
hsflowd: $(OBJS_HSFLOWD) $(HEADERS)
	$(CC) $(CFLAGS) -o $@ $(OBJS_HSFLOWD) $(LIBS_HSFLOWD) $(LDFLAGS_HSFLOWD) -static

执行重新执行命令make，发现会弹出告警信息类似于如下，目前运行暂时没发现有什么问题

/usr/bin/ld: evbus.o: in function `loadModule':
/home/gong/rust-work/github/host-sflow/src/Linux/evbus.c:213: 警告： Using 'dlopen' in statically linked applicati requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: hsflowd.o: in function `drop_privileges':
/home/gong/rust-work/github/host-sflow/src/Linux/hsflowd.c:1650: 警告： Using 'getpwnam' in statically linked appltions requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: util.o: in function `parseOrResolveAddress':
/home/gong/rust-work/github/host-sflow/src/Linux/util.c:952: 警告： Using 'getaddrinfo' in statically linked appliions requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: evbus.o: in function `loadModule':
/home/gong/rust-work/github/host-sflow/src/Linux/evbus.c:213: 警告： Using 'dlopen' in statically linked applicati requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: hsflowd.o: in function `drop_privileges':
/home/gong/rust-work/github/host-sflow/src/Linux/hsflowd.c:1650: 警告： Using 'getpwnam' in statically linked appltions requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: util.o: in function `parseOrResolveAddress':
/home/gong/rust-work/github/host-sflow/src/Linux/util.c:952: 警告： Using 'getaddrinfo' in statically linked appliions requires at runtime the shared libraries from the glibc version used for linking

编辑/etc/hsflowd.conf

sflow {
  collector { ip = 127.0.0.1 udpport = 6343 }
  polling = 3
  sampling = 10
  pcap { speed = 1- }
  tcp {}
}
sflow {
  collector { ip = 127.0.0.1 udpport = 6343 }
  polling = 3
  sampling = 10
  pcap { speed = 1- }
  tcp {}
}

配置参考

https://sflow.net/host-sflow-linux-config.php
https://sflow.net/host-sflow-linux-config.php

参数解释

  • collector { ip = 127.0.0.1 udpport = 6343 }：表示采集的数据发送到collector，对应的ip/port作为目标地址
  • polling：周期性统计一次，单位秒
  • sampling = 10：配置样本率，此处配置为1/10为样本
  • pcap { speed = 1- }: 表示监听所有网卡数据
  • tcp {}：监听tcp性能

运行

$ sudo ./src/Linux/hsflowd -ddd
$ sudo ./src/Linux/hsflowd -ddd

sFlow colletor端配置与使用 ​

依赖项目sflowtool作为collector端

https://github.com/sflow/sflowtool
https://github.com/sflow/sflowtool

编译

shell
$ git clone https://github.com/sflow/sflowtool.git
$ cd sflowtool
$ make
$ git clone https://github.com/sflow/sflowtool.git
$ cd sflowtool
$ make

如果需要静态链接库，可以查看make的输出，找到如下一行

gcc  -g -O2   -o sflowtool sflowtool.o
gcc  -g -O2   -o sflowtool sflowtool.o

可以看出是生成sflowtool的一步，所以修改一下，执行如下命令

shell
$ cd src
$ gcc  -g -O2   -o sflowtool sflowtool.o -static
$ cd src
$ gcc  -g -O2   -o sflowtool sflowtool.o -static

安装到/usr/bin以及systemd注册(这一步可以选择执行)

shell
$ sudo make install
$ sudo make install

开启监听端口6343（默认端口），-j表示输出json格式数据

shell
$ ./src/sflowtool -j
$ ./src/sflowtool -j

数据格式类似于

json
{
    "datagramSourceIP":"127.0.0.1",
    "datagramSize":"996",
    "unixSecondsUTC":"1692088861",
    "localtime":"2023-08-15T16:41:01+0800",
    "datagramVersion":"5",
    "agentSubId":"100000",
    "agent":"10.30.6.88",
    "packetSequenceNo":"19",
    "sysUpTime":"58105",
    "samplesInPacket":"1",
    "samples":[
        {
            "sampleType_tag":"0:2",
            "sampleType":"COUNTERSSAMPLE",
            "sampleSequenceNo":"19",
            "sourceId":"2:1",
            "elements":[
                {
                    "counterBlock_tag":"0:2001",
                    "adaptor_list":[
                        {
                            "ifIndex":"4",
                            "MACs":"1",
                            "mac_list":[
                                "525400e6b543"
                            ]
                        }
                },
                {
                    "counterBlock_tag":"0:2010",
                    "udpInDatagrams":"1043956",
                    "udpNoPorts":"1015",
                    "udpInErrors":"3",
                    "udpOutDatagrams":"44520",
                    "udpRcvbufErrors":"0",
                    "udpSndbufErrors":"0",
                    "udpInCsumErrors":"3"
                },
                {
                    "counterBlock_tag":"0:2009",
                    "tcpRtoAlgorithm":"1",
                    "tcpRtoMin":"200",
                    "tcpRtoMax":"120000",
                    "tcpMaxConn":"4294967295",
                    "tcpActiveOpens":"36045",
                    "tcpPassiveOpens":"16710",
                    "tcpAttemptFails":"1000",
                    "tcpEstabResets":"511",
                    "tcpCurrEstab":"42",
                    "tcpInSegs":"558012",
                    "tcpOutSegs":"551616",
                    "tcpRetransSegs":"10441",
                    "tcpInErrs":"22",
                    "tcpOutRsts":"674",
                    "tcpInCsumErrors":"1"
                },
                {
                    "counterBlock_tag":"0:2008",
                    "icmpInMsgs":"943",
                    "icmpInErrors":"0",
                    "icmpInDestUnreachs":"0",
                    "icmpInTimeExcds":"931",
                    "icmpInParamProbs":"0",
                    "icmpInSrcQuenchs":"0",
                    "icmpInRedirects":"0",
                    "icmpInEchos":"0",
                    "icmpInEchoReps":"12",
                    "icmpInTimestamps":"0",
                    "icmpInAddrMasks":"0",
                    "icmpInAddrMaskReps":"0",
                    "icmpOutMsgs":"0",
                    "icmpOutErrors":"0",
                    "icmpOutDestUnreachs":"1027",
                    "icmpOutTimeExcds":"0",
                    "icmpOutParamProbs":"1013",
                    "icmpOutSrcQuenchs":"0",
                    "icmpOutRedirects":"0",
                    "icmpOutEchos":"0",
                    "icmpOutEchoReps":"0",
                    "icmpOutTimestamps":"2",
                    "icmpOutTimestampReps":"12",
                    "icmpOutAddrMasks":"0",
                    "icmpOutAddrMaskReps":"0"
                },
                {
                    "counterBlock_tag":"0:2007",
                    "ipForwarding":"1",
                    "ipDefaultTTL":"64",
                    "ipInReceives":"1002278",
                    "ipInHdrErrors":"0",
                    "ipInAddrErrors":"0",
                    "ipForwDatagrams":"0",
                    "ipInUnknownProtos":"0",
                    "ipInDiscards":"0",
                    "ipInDelivers":"893313",
                    "ipOutRequests":"608419",
                    "ipOutDiscards":"4",
                    "ipOutNoRoutes":"18",
                    "ipReasmTimeout":"0",
                    "ipReasmReqds":"0",
                    "ipReasmOKs":"0",
                    "ipReasmFails":"0",
                    "ipFragOKs":"0",
                    "ipFragFails":"0",
                    "ipFragCreates":"0"
                },
                {
                    "counterBlock_tag":"0:2005",
                    "disk_total":"1457563774976",
                    "disk_free":"375918977024",
                    "disk_partition_max_used":"88.93",
                    "disk_reads":"592657",
                    "disk_bytes_read":"19298371584",
                    "disk_read_time":"186113",
                    "disk_writes":"1194763",
                    "disk_bytes_written":"30788962304",
                    "disk_write_time":"1103791"
                },
                {
                    "counterBlock_tag":"0:2004",
                    "mem_total":"33446776832",
                    "mem_free":"2456457216",
                    "mem_shared":"0",
                    "mem_buffers":"1418944512",
                    "mem_cached":"15454269440",
                    "swap_total":"19999485952",
                    "swap_free":"19997913088",
                    "page_in":"9562998",
                    "page_out":"15033689",
                    "swap_in":"12",
                    "swap_out":"348"
                },
                {
                    "counterBlock_tag":"0:2003",
                    "cpu_load_one":"0.990",
                    "cpu_load_five":"0.980",
                    "cpu_load_fifteen":"0.980",
                    "cpu_proc_run":"2",
                    "cpu_proc_total":"2466",
                    "cpu_num":"12",
                    "cpu_speed":"800",
                    "cpu_uptime":"26827",
                    "cpu_user":"16070320",
                    "cpu_nice":"8310",
                    "cpu_system":"5016330",
                    "cpu_idle":"295614880",
                    "cpu_wio":"189450",
                    "cpuintr":"0",
                    "cpu_sintr":"1341860",
                    "cpuinterrupts":"139569335",
                    "cpu_contexts":"243758350",
                    "cpu_steal":"0",
                    "cpu_guest":"0",
                    "cpu_guest_nice":"0"
                }
            ]
        }
    ]
}
{
    "datagramSourceIP":"127.0.0.1",
    "datagramSize":"996",
    "unixSecondsUTC":"1692088861",
    "localtime":"2023-08-15T16:41:01+0800",
    "datagramVersion":"5",
    "agentSubId":"100000",
    "agent":"10.30.6.88",
    "packetSequenceNo":"19",
    "sysUpTime":"58105",
    "samplesInPacket":"1",
    "samples":[
        {
            "sampleType_tag":"0:2",
            "sampleType":"COUNTERSSAMPLE",
            "sampleSequenceNo":"19",
            "sourceId":"2:1",
            "elements":[
                {
                    "counterBlock_tag":"0:2001",
                    "adaptor_list":[
                        {
                            "ifIndex":"4",
                            "MACs":"1",
                            "mac_list":[
                                "525400e6b543"
                            ]
                        }
                },
                {
                    "counterBlock_tag":"0:2010",
                    "udpInDatagrams":"1043956",
                    "udpNoPorts":"1015",
                    "udpInErrors":"3",
                    "udpOutDatagrams":"44520",
                    "udpRcvbufErrors":"0",
                    "udpSndbufErrors":"0",
                    "udpInCsumErrors":"3"
                },
                {
                    "counterBlock_tag":"0:2009",
                    "tcpRtoAlgorithm":"1",
                    "tcpRtoMin":"200",
                    "tcpRtoMax":"120000",
                    "tcpMaxConn":"4294967295",
                    "tcpActiveOpens":"36045",
                    "tcpPassiveOpens":"16710",
                    "tcpAttemptFails":"1000",
                    "tcpEstabResets":"511",
                    "tcpCurrEstab":"42",
                    "tcpInSegs":"558012",
                    "tcpOutSegs":"551616",
                    "tcpRetransSegs":"10441",
                    "tcpInErrs":"22",
                    "tcpOutRsts":"674",
                    "tcpInCsumErrors":"1"
                },
                {
                    "counterBlock_tag":"0:2008",
                    "icmpInMsgs":"943",
                    "icmpInErrors":"0",
                    "icmpInDestUnreachs":"0",
                    "icmpInTimeExcds":"931",
                    "icmpInParamProbs":"0",
                    "icmpInSrcQuenchs":"0",
                    "icmpInRedirects":"0",
                    "icmpInEchos":"0",
                    "icmpInEchoReps":"12",
                    "icmpInTimestamps":"0",
                    "icmpInAddrMasks":"0",
                    "icmpInAddrMaskReps":"0",
                    "icmpOutMsgs":"0",
                    "icmpOutErrors":"0",
                    "icmpOutDestUnreachs":"1027",
                    "icmpOutTimeExcds":"0",
                    "icmpOutParamProbs":"1013",
                    "icmpOutSrcQuenchs":"0",
                    "icmpOutRedirects":"0",
                    "icmpOutEchos":"0",
                    "icmpOutEchoReps":"0",
                    "icmpOutTimestamps":"2",
                    "icmpOutTimestampReps":"12",
                    "icmpOutAddrMasks":"0",
                    "icmpOutAddrMaskReps":"0"
                },
                {
                    "counterBlock_tag":"0:2007",
                    "ipForwarding":"1",
                    "ipDefaultTTL":"64",
                    "ipInReceives":"1002278",
                    "ipInHdrErrors":"0",
                    "ipInAddrErrors":"0",
                    "ipForwDatagrams":"0",
                    "ipInUnknownProtos":"0",
                    "ipInDiscards":"0",
                    "ipInDelivers":"893313",
                    "ipOutRequests":"608419",
                    "ipOutDiscards":"4",
                    "ipOutNoRoutes":"18",
                    "ipReasmTimeout":"0",
                    "ipReasmReqds":"0",
                    "ipReasmOKs":"0",
                    "ipReasmFails":"0",
                    "ipFragOKs":"0",
                    "ipFragFails":"0",
                    "ipFragCreates":"0"
                },
                {
                    "counterBlock_tag":"0:2005",
                    "disk_total":"1457563774976",
                    "disk_free":"375918977024",
                    "disk_partition_max_used":"88.93",
                    "disk_reads":"592657",
                    "disk_bytes_read":"19298371584",
                    "disk_read_time":"186113",
                    "disk_writes":"1194763",
                    "disk_bytes_written":"30788962304",
                    "disk_write_time":"1103791"
                },
                {
                    "counterBlock_tag":"0:2004",
                    "mem_total":"33446776832",
                    "mem_free":"2456457216",
                    "mem_shared":"0",
                    "mem_buffers":"1418944512",
                    "mem_cached":"15454269440",
                    "swap_total":"19999485952",
                    "swap_free":"19997913088",
                    "page_in":"9562998",
                    "page_out":"15033689",
                    "swap_in":"12",
                    "swap_out":"348"
                },
                {
                    "counterBlock_tag":"0:2003",
                    "cpu_load_one":"0.990",
                    "cpu_load_five":"0.980",
                    "cpu_load_fifteen":"0.980",
                    "cpu_proc_run":"2",
                    "cpu_proc_total":"2466",
                    "cpu_num":"12",
                    "cpu_speed":"800",
                    "cpu_uptime":"26827",
                    "cpu_user":"16070320",
                    "cpu_nice":"8310",
                    "cpu_system":"5016330",
                    "cpu_idle":"295614880",
                    "cpu_wio":"189450",
                    "cpuintr":"0",
                    "cpu_sintr":"1341860",
                    "cpuinterrupts":"139569335",
                    "cpu_contexts":"243758350",
                    "cpu_steal":"0",
                    "cpu_guest":"0",
                    "cpu_guest_nice":"0"
                }
            ]
        }
    ]
}

参考阅读 ​

什么是sFlow？-华为

sFlow sampled flow协议解析

sFlow介绍

Configuring Host sFlow for Linux via /etc/hsflowd.conf

Last updated:

Released under the MIT License.

Copyright © 2023 gongzhengyang