how to use the self-made network monitor of raspberry pie

Posted by millikan at 2020-03-15

The tool described in this article is suitable for "black box testing" in a home environment. It can help you record everything that happens in the network. You can use it to detect network threats, or provide data to relevant experts for network forensic analysis.

If you need a solution in an enterprise environment, you can refer to [this article] of security onion.

Shopping list

1. Raspberry Pie 3 (shell + power + circuit board)

2. SanDisk class 10 microSD card 64GB (80mb / s)

3.      Debian OS-Linux RaspbianLite

4. I use d-link1100-08p for network Gigabit switches or other devices that support port mirroring

5.      Critical Stack API(Threat Intel/ IOCs)

6. Mailgun account or similar email service supporting alert / notification

Total value: about £ 75

Overview map

Introduction to key technologies

What is port mirroring?

Copying traffic data from one port to another (passive) will increase the operating load of the switch.

What is bro?

An IDS protocol analysis tool, you can use it as Wireshark's protocol analyzer, but it has no GUI interface and is faster.

What is netsniff?

The daemons for packet capture use AF packet to speed up packet capture.

What is Loki?

Yara based file scanning daemons. Some are similar to anti-virus products based on signature detection, but you can make your own detection rules.

What is critical stack?

A threat intelligence platform that you can link to through the API on raspberry pie.

What is team Cymru MHR?

A malware hash database where you can use the information to match the detected malware hash.

Start working

1. Brush the system into raspberry pie

2. Assign an IP to raspberry pie

3. Run bash script

4. Get it done

1、 Swipe raspbian into the microSD card

I use MacBook, so different platforms may have different methods. Users of other platforms can refer to [this tutorial] for more information.

Insert microSD card:

diskutil list

Disk number found:

diskutil unmountDisk /dev/disk<disk#from diskutil>

Brush the raspbian image into the microSD card 'disk':

sudo dd bs=1m if=image.imgof=/dev/rdisk<disk# from diskutil> diskutil unmountDisk /dev/disk<disk#from diskutil>

2、 Configure network

Log in using the default configuration. User name: Pi, password: raspberry.

Set the IP of WLAN 0 (WiFi) for trusted management access:

sudo nano/etc/wpa_supplicant/wpa_supplicant.conf network={    ssid="The_ESSID_from_earlier"    psk="Your_wifi_password" }  sudo ifdown wlan0 sudo ifup wlan0 ifconfig wlan0

After you get a DHCP IP, you can use SSH to access this node. Next, leave eth0 as the mirror interface, which does not need to set the IP address.

sudo apt-get update && sudo apt-get-y install vim sudo vim /etc/network/interfaces

Add the following code:

iface eth0 inet static static ip_address= sudo ifconfig eth0 down && sudoifconfig eth0 up

Three, deployment

Download and run the bash script, which has been successfully tested on raspbian.

-The core components of the installer

-Configure network options (disable NIC offloading)

-Create services for each program

-Creating mail alerts using mailgun / ssmtp

-Configure cron tasks

[email protected]:~# sudo su - [email protected]:~# apt-get install -y git [email protected]:~# cd ~ [email protected]:~# git clone [email protected]:~# chmod +xfoxhound-nsm/ [email protected]:~# ./foxhound-nsm/ [email protected]:~ $ sudo tcpreplay -t -K -q--loop=10 --intf1=eth0 /opt/foxhound-1476541443.pcap Actual: 1048570 packets (1050923190 bytes)sent in 87.62 seconds.           Rated:11994102.0 bps, 91.51 Mbps, 11967.25 pps

Here are the broctl netstats data before and after sending:

[email protected]:/etc/network# broctl netstats        bro: 1476547903.768150 recvd=1951368 dropped=5408 link=1956776 [email protected]:/etc/network# broctl netstats        bro: 1476548144.248161 recvd=3012168 dropped=14608 link=3026776

For a home or lab environment, its performance is relatively good (only 10000 packets are discarded from a million packets). My bandwidth speed is 40mbps, which is enough for IDS system. If you still need to improve performance, it is recommended to consider replacing raspberry pie.

If you want more NIC perf tests, you can run the following commands on the server:

[email protected]:~$ sudo apt install iperf3 [email protected]:~$ iperf3 -s ----------------------------------------------------------- Serverlistening on 5201

Then run on raspberry Pie:

[email protected]:~# apt install iperf3 [email protected]:~# iperf3 -c -i 1 -t20 Connecting to host, port 5201 [  4]local port 38562 connected to port 5201 [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd [ 4]   0.00-1.00   sec 8.86 MBytes  74.3 Mbits/sec    0  89.1 KBytes       ... ... ...      [ 4]  19.00-20.00  sec 9.26 MBytes  77.7 Mbits/sec    0  1.23 MBytes       - - - - - - - - - - - - - - - - - - - - - -- - - [ ID] Interval           Transfer     Bandwidth       Retr [ 4]   0.00-20.00  sec  185 MBytes  77.5 Mbits/sec  139             sender [ 4]   0.00-20.00  sec  184 MBytes  77.1 Mbits/sec                  receiver   iperf Done.

I assume that the bottleneck here is the microSD card, whose speed is about 80mb / s.

Bro Foundation

-The default storage path for all bro logs is / NSM / bro / logs/

-The default script path is / usr / local / bro / share / bro / site / bro scripts/

The log directory structure is roughly as follows:

[email protected]:/nsm/bro/logs/current $ ls-lash total 6.9M 4.0K drwxr-xr-x 3 root root  4.0K Oct 15 16:11 . 4.0K drwxr-xr-x 5 root staff 4.0K Oct 1516:50 .. 4.0K -rw-r--r-- 1 root root   349 Oct 15 16:51 app_stats.log 4.0K -rw-r--r-- 1 root root   121 Oct 15 15:51 .cmdline  16K-rw-r--r-- 1 root root   14K Oct 15 16:30communication.log 2.9M -rw-r--r-- 1 root root  2.9M Oct 15 16:52 conn.log  16K-rw-r--r-- 1 root root   14K Oct 15 16:52dhcp.log 384K -rw-r--r-- 1 root root  379K Oct 15 16:52 dns.log 4.0K -rw-r--r-- 1 root root   345 Oct 15 15:51 .env_vars 1.2M -rw-r--r-- 1 root root  1.2M Oct 15 16:52 files.log 1.6M -rw-r--r-- 1 root root  1.6M Oct 15 16:52 http.log 4.0K -rw-r--r-- 1 root root   291 Oct 15 16:44 known_hosts.log 4.0K -rw-r--r-- 1 root root   327 Oct 15 16:34 known_services.log  12K-rw-r--r-- 1 root root   11K Oct 15 16:50notice.log 4.0K -rw-r--r-- 1 root root     5 Oct 15 15:51 .pid 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.communication 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.conn 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:01 .rotated.conn-summary 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.dhcp 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.dns 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.files 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.http 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.known_hosts 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.known_services 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.loaded_scripts 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.notice 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.packet_filter 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.ssl 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.weird 4.0K -rw-r--r-- 1 root root    18 Oct 15 16:00 .rotated.x509 4.0K -rw-r--r-- 1 root root  3.0K Oct 15 16:51 software.log 320K -rw-r--r-- 1 root root  314K Oct 15 16:52 ssl.log 4.0K -rw-r--r-- 1 root root    58 Oct 15 15:51 .startup 4.0K drwx------ 3 root root  4.0K Oct 15 15:51 .state 4.0K -rwx------ 1 root root    18 Oct 15 15:51 .status 4.0K -rw-r--r-- 1 root root    46Oct 15 15:51 stderr.log 4.0K -rw-r--r-- 1 root root   188 Oct 15 15:51 stdout.log  24K-rw-r--r-- 1 root root   17K Oct 15 16:52weird.log 416K -rw-r--r-- 1 root root  412K Oct 15 16:52 x509.log

You can use the head command to find the text field name:

[email protected]:$ head  dns.log #separator \x09 #set_separator         , #empty_field   (empty) #unset_field    - #path        dns #open       2016-10-15-16-00-01 #fields      ts      uid   id.orig_h  id.orig_p  id.resp_h id.resp_p         proto        trans_id   query        qclass       qclass_name         qtype        qtype_name    rcode        rcode_name    AA    TC         RD    RA    Z       answers   TTLs rejected #types      time string        addr port addr port enum        count         string        count        string        count        string        count         string        bool bool bool bool count        vector[string]         vector[interval]        bool

Use bro cut to resolve logs:

[email protected]:$ cat dns.log | bro-cut -D tsid.orig_h id.orig_p id.resp_h id.resp_p proto query answers TTLs

Basic report:

[email protected]:$ bro-cut query < dns.log |sort | uniq -c | sort -rn | head -n 10 tail -f dns.log | awk  '{print $3, $7, $9}'

Note that only conn.log can enable maxmind geoip queries.

[email protected]:$ bro-cut resp_cc < conn.log| sort | uniq -c | sort -rn | head -n 10    755 US    524 RO    123 GB     49 NL      28 EU     25 IE     10 DE      7 ES      6 CA

I haven't made a dashboard yet. If you don't want to do it yourself, you can directly use VPS elk instance to store all logs in the cloud, or you can use elk / Splunk / graylog to store logs locally.

Foundation maintenance

To start / stop netsniff ng:

[email protected]:~ $ sudo service netsniff-ngstop [email protected]:~ $ sudo service netsniff-ngstart [email protected]:~ $ sudo service netsniff-ngstatus

On / off bro:

[email protected]:~ $ sudo -i broctl stop [email protected]:~ $ sudo -i broctl start [email protected]:~ $ sudo -i broctl netstats [email protected]:~ $ sudo -i broctl status

Run Loki manually:

[email protected]:~ $ python /nsm/Loki/ --dontwait --onlyrelevant -p /nsm/bro/extracted -l /nsm/Loki/log [email protected]:~ $ sudo -u critical-stackcritical-stack-intel list

Reference material

1. Deploy network security monitoring:

2. Network security monitoring practice:

3. Laika boss object scanning system:

4.      PassiveDNS:

5. D3js (graphical tool):

6.      Graylog:

*Reference source: sneakymonkey, compiled by FB editor alpha? H4ck, reprinted from