basic log server for detecting attacks part 1: server settings

Posted by trammel at 2020-03-10

Words written in the front

At the beginning of this year, I had an idea to improve my infrastructure log management system. But up to now, my log management technology is only limited to opening the terminal, connecting SSH, then processing my infrastructure assets, and tracking the log files that I am interested in. But it wasn't until I read this great blog post by Jeff dimcock and Steve borosh. A small voice in my head has been nagging me about what I should do.

This blog series will serve as a guide for setting up centralized infrastructure logging, monitoring, and alerting. But it's a very broad topic, I can't cover it all, but I still hope to provide some help to people in need.

Main contents of this series:

Part 1: logging server settings

Part 2: log aggregation

Part 3: graylog dashboard 101

Part 4: log event alerts

At the end of this series, we will finally create the logging settings shown in the following figure:

To be honest, regular penetration testing doesn't require much effort to invest in infrastructure or log management. However, if you are involved in long-term (months / years) infrastructure projects, you should invest more time in setting up centralized logging. I think there are the following reasons:

1. Monitor operations - centralized logging allows you to view ongoing operations at any time: successful phishing, payload downloads, potential event response activities, attacks on your assets, and more. With this oversight, you can react immediately to events and even adjust your tactics. For example, your log alert tells you that your payload has been downloaded 10 times in the past 8 hours, and you haven't got a shell yet.

2. Reporting - good logging improves the quality of reporting.

3. Improve convenience and efficiency - monitoring logs from multiple infrastructure assets is painful. Configuring custom quick statistics and alerts saves me a lot of time and effort.

4. Accountability - you should know and have proof of the task you are responsible for.

5. Security - because the Internet is dark and full of fear. The blue team is monitoring their infrastructure logs for signs of anomalies and malicious activity, why can't you?

Set up logging server


I have several reasons why I don't use plenty instead of graylog2 as a centralized blog logging server for the entire series,

1. It is open source, and the daily log volume is less than 5GB. This is enough for a normal pentester / red tea mer.

2. The recording function is really good.

3. It has many out of the box functions. If you want to add functions, it has many additional plug-ins to choose from.

4. It supports slack alert.

I'll show you how to set up a new graylog logging server properly.

1. Server requirements:

Graylog has several conditions, and I'll cover the installation. A server itself, although graylog is built on elastic search, only needs 2GB of ram to run. If you want to have a better experience, I recommend using 4GB ram, and graylog's documentation covers the installation of various operating systems. I will use

A demonstration of the new Debian 9 system.

2. Prerequisites:

Graylog has the following dependencies:

Java(> = 8)

MongoDB(> = 2.4)

Elasticsearch(> = 2.x)

Let's start the installation: javasudo apt update & & sudo apt upgradesdo apt install apt transport HTTPS openjdk-8-jre-headless UUID runtime pwgen

sudo apt update && sudo apt upgrade
sudo apt install apt-transport-https openjdk-8-jre-headless uuid-runtime pwgen

#MongoDBsudo apt-key adv --keyserver hkp:// --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5echo "deb jessie/mongodb-org/3.6 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.listsudo apt-get update && sudo apt-get install -y dirmngr mongodb-org

sudo apt-key adv --keyserver hkp:// --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
echo "deb jessie/mongodb-org/3.6 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
sudo apt-get update && sudo apt-get install -y dirmngr mongodb-org

Note: if installation of mongodb fails, you may want to install libssl1.0.0 package. Add Debian's Jessie backports to your / etc / apt / sources.list:

# Jessie backportsdeb jessie-backports main

# Jessie backports
deb jessie-backports main

Or you can download and install the missing dependencies yourself.

The next dependent installation is elasticsearch.


#Elasticsearch wget -qO - | sudo apt-key add -

echo“deb stable main”| sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

echo“deb stable main”| sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

sudo apt-get update && sudo apt-get install elasticsearch

sudo apt-get update && sudo apt-get install elasticsearch sudo systemctl daemon-reload

sudo systemctl enable mongod.service elasticsearch.service

sudo systemctl enable mongod.service elasticsearch.service
sudo systemctl restart mongod.service elasticsearch.service

3.安装Graylog2: Graylog提供DEB和RPM包存储库。wget dpkg -i graylog-2.4-repository_latest.debsudo apt-get updatesudo apt-get install graylog-server

Graylog提供DEB和RPM包存储库。 wget sudo dpkg -i graylog-2.4-repository_latest.deb sudo apt-get update sudo apt-get install graylog-server

4. Configure graylog2:

All graylog configurations are managed from a single file; / etc / graylog / server / server.conf. Before we log in to graylog's web administration page, we need to change some of these settings.

Remember to back up the configuration file before you do something, because you never know when to start over.

sudo cp /etc/graylog/server/server.conf /etc/graylog/server/server.conf.bak

sudo cp /etc/graylog/server/server.conf /etc/graylog/server/server.conf.bak

1) Administrator user name:

You can change the user name of the administrator, which is "admin" by default.

root_username = admin

root_username = admin pwgen -N 1 -s 96

Paste the entire string into the password? Secret setting:

password_secret = GENERATED_SECRET

password_secret = GENERATED_SECRET echo -n yourpassword | sha256sum

Paste the hash into the root password Sha2 setting: root password Sha2 = password hash

root_password_sha2 = PASSWORD_HASH

4) Network listening port:

Finally, if you want to use any port other than the default 9000, you should enable the web interface and change its listening port. Uncomment the following lines and change them to match the web interface of graylog for any port you want to access:

rest_listen_uri =

rest_listen_uri =
web_listen_uri =

Then you just need to enable and restart the graylog service.

Sudo systemctl daemon reloadsudo systemctl enable graylog-server.servicesudo service graylog-server restart log in to your web management interface http: / / [ip_address]: 9000/

sudo systemctl daemon-reload
sudo systemctl enable graylog-server.service
sudo service graylog-server restart
http:// [IP_ADDRESS]:9000 /

Important: the next step is optional, but I recommend not skipping it. Graylog's web interface and rest API use HTTP by default, which means your password and other sensitive data are sent in clear text. The next step involves generating and adding a self signed HTTPS certificate for your graylog installation.

4) Install self signed certificate:

Create a folder to manage your certificates to it:

sudo mkdir / etc / graylog / server / ssl

sudo mkdir / etc / graylog / server / ssl

cd / etc / graylog / server / ssl

cd / etc / graylog / server / ssl

Create the file "openssl-graylog.cnf" and fill in the following; customize it to meet your needs:



distinguished_name = req_distinguished_name

distinguished_name = req_distinguished_name

x509_extensions = v3_req

x509_extensions = v3_req
prompt = no

More about certificate issuer


C = US

C = US

L = NY

L = NY

this is it.

O = Graylog

Or =Graylog

OU = Graylog

CN =

CN =




keyUsage = keyEncipherment,dataEncipherment

extendedKeyUsage = serverAuth

extendedKeyUsage = serverAuth

The name.

subjectAltName = @alt_names

The IP address and DNS name that the certificate should contain, the IP address and DNS IP address, DNS name, and "," are consecutive numbers. [alt_names]IP.1 = =

#证书应包含的IP地址和DNS名称#IP 地址和DNS的 IP地址### DNS名称的 ###,“###”是连续的数。
IP.1 =
DNS.1 =

Note: make sure that the 'IP. 1 =' value in the configuration file is changed to the IP address of the graylog server.

注: 请确保将配置文件中的'IP.1 ='值更改为Graylog服务器的IP地址。


SUDO Openssl Req-X509-days 365-nodes-Newkey RSA:2048-conging opensl-graylog.cnf-keyout PKCS5-plain.pem-out graylog.crt

sudo openssl req -x509 -days 365 -nodes -newkey rsa:2048 -config openssl-graylog.cnf -keyout pkcs5-plain.pem -out graylog.crt

Transfer your PKCs 5 private key to the unencrypted PKCs 8 private key (graylog). Key):

sudo openssl pkcs8 -in pkcs5-plain.pem -topk8 -nocrypt -out graylog.key

sudo openssl pkcs8 -in pkcs5-plain.pem -topk8 -nocrypt -out graylog.key

When using HTTPS's graylog rest API, the X.509 certificate (graylog, CRT in this case) must be trusted by the JVM trust store, otherwise communication will fail. Since we do not want to be confused with the official trust repository, we will make a copy for use with our graylog certificate.

sudo cp -a / usr / lib / jvm / java-8-openjdk-amd64 / jre / lib / security / cacerts / etc / graylog / server / ssl /sudo keytool -importcert -keystore / etc / graylog / server / ssl / cacerts -storepass changeit -alias graylog-self-signed -file /etc/graylog/server/ssl/graylog.crt

sudo cp -a / usr / lib / jvm / java-8-openjdk-amd64 / jre / lib / security / cacerts / etc / graylog / server / ssl /
sudo keytool -importcert -keystore / etc / graylog / server / ssl / cacerts -storepass changeit -alias graylog-self-signed -file /etc/graylog/server/ssl/graylog.crt

The certificate should now be available. Edit graylog's configuration file (/ etc / graylog / server / server. CONF) and locate and change the following settings:

#Rest API settings


Rest=Enable=U TLS=TRUE

rest_enable_tls = true

rest of the cert file = /etc/graylog/server/ssl/graylog.crt

rest_tls_cert_file = /etc/graylog/server/ssl/graylog.crt

rest of the fatty key file = /etc/graylog/server/ssl/graylog.key

rest_tls_key_file = /etc/graylog/server/ssl/graylog.key

#Web interface settings


web_enable_tls = true

web_enable_tls = true
web_tls_cert_file = / etc / graylog / server /ssl/graylog.crt


web_tls_key_file = /etc/graylog/server/ssl/graylog.key

Note: for system users running the graylog process, the certificate and key files need to be readable (644 permissions work for both files).

We're done! Simply restart graylog and you should be able to log in to your administration console at HTTPS: / / [ip_address]: 9000/

https:// [IP_ADDRESS]登录到您的管理控制台:9000 /

sudo service graylog-server restart

sudo service graylog-server restart

If you have any login problems, refer to the HTTPS installation documentation for graylog 2 and the log file for graylog (/ var / log / graylog-server / server. Log) for troubleshooting.


If you install it completely, you may find it a bit troublesome to install graylog, so I wrote a script to automatically perform all the above installation steps.

Protect graylog:

You should understand the risk of centralizing all logs from the attack infrastructure in one place. The more logs you aggregate, the higher the risk the log server carries; a compromise may expose your entire operation.

The following table shows the default listening ports for graylog:

Some simple firewall rules can protect your graylog installation, especially if you use a VPN server to control access to the management port of the attack infrastructure.

Here are some examples of iptables rules that you can apply to your graylog server to limit its attack surface.

Default policy







Allow connection



-A INPUT -m状态--state RELATED,ESTABLISHED -j ACCEPT #允许来自本地环回接口的流量

Input - I lo-j accept

-A INPUT -i lo - j ACCEPT

Only SSH connections from specific IP addresses, such as VPN, are allowed

-A INPUT -s [ VPN_IP_ADDRESS ] / 32 -p tcp -m tcp --dport 22 -j ACCEPT

Only allowed to connect to graylog management from a specific IP address, such as VPN

-A INPUT -s [ VPN_IP_ADDRESS ] / 32 -p tcp -m tcp --dport 9000 -j ACCEPT

Only rsyslog communication from attacking infrastructure assets is allowed (1 line per asset)

-A INPUT -s [ ASSET_IP_ADDRESS ] / 32 -p tcp -m tcp --dport 5140 -j ACCEPT

Enter - s [asset \ \ u address] / 32-p TCP-M TCP --- dport 5140-j accept

-A INPUT -s [ ASSET_IP_ADDRESS] / 32 -p tcp -m tcp --dport 5140 -j ACCEPT

Enter - s [asset \ \ u address] / 32-p TCP-M TCP --- dport 5140-j accept

-A INPUT -s [ ASSET_IP_ADDRESS ] / 32 -p tcp -m tcp --dport 5140 -j ACCEPT

Enter - s [asset \ \ u address] / 32-p TCP-M TCP --- dport 5140-j accept

-A INPUT -s [ ASSET_IP_ADDRESS ] / 32 - p tcp -m tcp --dport 5140 -j ACCEPT

Note: the last part of the above ruleset will be covered in detail in the next article.


Our logging server is up and running, and the next article will show you how to set up a rollup of logs from various infrastructure assets.