random bomb cyberspace x-search

Posted by tetley at 2020-02-24

I did not participate in the development of a set of cyberspace search engine, just used. After the advent of censys, I studied a little bit about their open source. So I'm interested in this kind of system and have been paying attention to it. Combined with Shodan's actions in the past two years, I've been playing around with my own thoughts. The big guys tap.

Shodan recently launched another product, Shodan monitor, which is essentially an external network asset monitoring service. It can be used for payment:

"Keep track of the devices that you have exposed to the Internet. Setup notifications, launch scans and gain complete visibility into what you have connected". The usage is that the customer provides a batch of IP exposed by his own external network, and Shodan is responsible for monitoring these IP (network assets). The monitoring items include but are not limited to:

Detect data leaks to the cloud;

phishing websites;

compromised databases。

It is also speculated that there are regular port monitoring, vulnerability monitoring, and other features that can be detected after compromised. Shodan monitor only needs the public network IP provided by the customer, and all other matters shall be handled by Shodan:

Their paid services are divided into three levels. The biggest difference between the three packages is the number of IP that supports monitoring (as shown in the figure below). With Shodan's infrastructure and technology, it's not difficult to do this. It can even be said that based on their cyberspace search engine, it's natural to do this.

With the accumulation of infrastructure and data, there are endless ways to play. Occasionally, I would listen to people in the circle. What's new about safety Please refer to Shodan.

Last year, Shodan released a byproduct of cyberspace search engine: malware hunter

A brief introduction on the official website explains the principle of the malware Hunter: "malware hunter is a specialized Shodan crawler that explores the Internet looking for Command & Control (C2S) servers for botnets. It does this by anticipating to be an affected client that's reporting back to a C2. Since we don't know where the C2S are located the crawler effectively reports back to every IP on the Internet as if the target IP is a C2. If the crawler gets a positive response from the IP then we know that it’s a C2.」

To be clear, this product is also based on the existing network wide asset retrieval ability of Shodan, plus some botnet communication features or a small amount of C2 features provided by our own research or partners, pretending to be BOT and C2 for communication, and the successful interaction is confirmed as C2. Not long ago, recorded future also issued a report, which introduced that during the period from December 2, 2018 to January 9, 2019, a large number of C & C servers of 14 rat (remote access tool, i.e. "remote control") were tracked with the project of "Shodan malware Hunter", and combined with the inspection data of their own products on the aspect of vicim, a score was sorted out Analysis report:

Among them, 14 malicious families are involved:

Bozok RAT








Xtreme RAT

DarkTrack RAT

Nuclear RAT



Orcus RAT

These practices of Shodan can be done by the search engine based on cyberspace. For example, two similar works of censys this year:

Hunting Mirai Control Servers Using Known Shell Scripts

Hunting for Threats:

 Coinhive Cryptocurrency Miner

For another example, Heige published two articles at the beginning of this year, introducing similar things based on zoomeye:

Identifying Cobalt Strike team servers in the wild by using ZoomEye

Hunting Botnet(Mirai .etc) Control Servers by using ZoomEye

In addition, more similar things can be done. As long as you analyze the relevant characteristics, you can retrieve them all over the network, and the direct output is high-quality threat intelligence data. For example, if we do the following two articles based on the whole network space search engine, we will get twice the result with half the effort:

Abuse of hidden “well-known” directory in HTTPS sites

Plugin vulnerabilities exploited in traffic monetization schemes

In the past, censys did a great job: it basically opened all the core components of its own cyberspace search engine (except database), including zmap / zgrab / ztag, and also sent a paper to explain the architecture and some technical details of censys:

Official website and components: open data:

Paper of the introduction of the censys architecture and technology:

Based on the open-source components and architecture paper of censys, in theory, I can develop a set of cyberspace search engine according to huluduapiao. Of course, there are many holes to be stepped on in the project. If it's too much trouble, we can use these open systems to retrieve, and their open retrieval data is also very practical.

In fact, it is a routine operation to search the whole network according to certain characteristics. Even, what can be retrieved in the whole network is not only the network asset retrieval that has been played by Shodan / zoomeye / cells / fofa, but also other features to retrieve other things in the whole network. For example, atuin system of Tencent Xuanwu laboratory, based on the characteristics of software and vulnerabilities, searches the existence of vulnerabilities throughout the network. Therefore, the title of this paper does not directly write "asset retrieval of cyberspace", but replaces "asset" with a letter "X", because the whole cyberspace can be retrieved on a large scale by similar means, not only the so-called "equipment assets".

I only have a perceptual understanding of atuin system, so I won't go into details, just look at their official introduction:

World Internet Conference leading scientific and technological achievements Tour:

"Artuin" system of Tencent security Xuanwu laboratory was selected

It took us four years to raise a turtle capable of exploding watches

Based on certain characteristics of the whole network retrieval method, should be more than these. It's a matter of testing brain holes. Fellow tycoons may as well use their brains a lot. They are not afraid to play with flowers, but they are afraid to stop.

Recently, we created a knowledge planet called network security stack.

If you search the knowledge planet, you will find many online security themed planets. Of course, there are more water, more water, sharing information is either not interested or shallow. There are also some, which are not without dry goods, even with high density of dry goods, but eventually become mechanical data sharing and technology post forwarding. The members of the planet just share various URLs one by one, with few heuristic discussions, experience summary of grounding gas, and brain opening with a flash of inspiration

I think the most recommended knowledge planet of network security technology is phith0n's code audit, eggplant bull's red blue confrontation, QZ security intelligence analysis and cosine's grey robe skills. It's just that code audit is still active and output dry goods I am interested in reverse analysis, botnet confrontation, threat intelligence analysis, and other security technology related directions, such as red blue confrontation and security data processing. I also read a lot of materials everyday, and I also have some ideas that I want to write down or share, and I also want to discuss these ideas with my peers Think about it, just create a knowledge planet by yourself.

To create this planet, the first step is to share some interesting technologies, skills, safety products, industries and other relevant thoughts seen elsewhere. It's also good just as a record of your own thinking. No hurry, no hurry, no interruption. But I think it's better to have a discussion together after I write too much. So now let's go public and hope that interested people join in, share some technologies that you find interesting, and share some thoughts about network security technology, products and industry.

The QR code is as follows, which can be added by scanning. I've set a payment threshold. If my wechat friends are interested in it, they can talk to me privately, and I'll be a guest in the circle. If not, I'm sorry to ask you to spend money. I hope you can share more and participate in the discussion after coming in, so as to make more money.