the life and death of cyberspace mapping (3)

Posted by tzul at 2020-02-28

The first two chapters talked about the overall thinking of some cyberspace mapping. Cyberspace mapping technology is to draw network maps, which will use many different scenes. For example, the impact of a vulnerability on the world, such as the analysis of cameras that can access video streams publicly in the world, the description of global data leakage (such as citizen data), and the ranking of industrial control system models in specific regions. It is estimated that people are more likely to yawn. "What is it? It's not something normal people can do. It's not worth it at all.". I don't deny this conclusion. It really has no significance. Normal people can't do these things. Aren't the key "enemies" all abnormal people? These things will not produce any value to human beings, neither promote the progress of human science, nor give us more bread. However, if it is used for sabotage, it is a necessary medicine for home travel.

The rule of network survival is similar to the rule of dark forest. If you want to survive safely, you need to hide yourself, because "once found, you will be hit.". Being attacked is inevitable, and not being attacked is an extremely accidental exception. In theory, there is no system that can't be attacked. Being attacked is only a matter of time, and only costs and benefits are considered. So when we track and find that the machine of an important unit is used to dig a pit, and the banking system of a country is sold as a shell with a few cents, we are surprised to find that the attacker does not realize the importance of the accused system at all, which belongs to the "easy" behavior. Since it is inevitable to be attacked and there are loopholes, how can enterprises and institutions measure the effectiveness of protection? In the early days of barbaric growth, based on the compliance logic, the most original way to judge is to establish a product protection system, such as buying a firewall IDS scanner. With the continuous upgrading and improvement of technology, we are no longer as we used to be, so we have put forward a higher and more scientific method to judge whether it can be effective for important systems (key information Information infrastructure), including data storage security and business operation stability.

It is impossible for network protection to eliminate loopholes. If we set the goal like this, we will be in endless pain. On the premise that we have done a lot of basic data analysis and event tracking, we have summed up a simple formula to evaluate the effective value of safety protection:

p = 1 / ( a * t )

Where p represents the effective value of protection, a represents the attack surface, and T represents the average time of vulnerability response. It can be seen that the effective value of protection is inversely proportional to the attack surface, and the more the attack surface, the lower the effective value of protection, and vice versa; at the same time, the effective value of protection is inversely proportional to the average time of vulnerability response, and the longer the response time, the worse the protection effect. A friend here may ask, is it not related to the number of loopholes? Nothing is wrong, but it doesn't matter much, because first, the number of vulnerabilities is equal to everyone, and second, in front of 0day, the best way is to reduce the attack area. So we think that the weight in the evaluation formula is ignored. Even if there are no known vulnerabilities at present, it doesn't mean that your protection effect is 100%. Because your attack surface is always there, the new vulnerabilities will be updated every day, so what the protection party has to do is to constantly improve its response speed.

This is exactly what happened to the attackers. We used to assume that the attacker was targeted. He knows who you are and he knows to get your sensitive data. That's right, it used to be, because the best way for hackers to get the lowest cost and the most profit is to target important targets. Then hackers found two big problems: first, for a specific point in time, I may not have an available and effective vulnerability to take down the target system directly, so I can only keep it for a long time; second, I may not have a relatively complete IP list against the target, because the initial approach is a vulnerability to the main station for hard connection, and then The master station can not be hard wired down. It can also be used for small systems that are not so concerned. After all, the intranet is interconnected. Based on the consideration of input-output ratio, hackers have split again. Some people continue to focus on important information systems. Although the cost is very high, the difficulty directly leads to the rise of a single income. Some people think that important information systems are not easy to handle. I can knead the soft persimmon, and "small profits and more sales" can also make money, so they come up with several kinds of ideas The new model: from controlling the black chain diversion of broilers at first, to controlling the DDoS traffic of large-scale broilers, to the extortion software that encrypts data (your data is useless to me, but it is useful to you). Now it develops into controlling the virtual currency of broilers mining.

By the time these models come out, you can already see that it doesn't matter who the goal is. Mass network wide strike without specific targets is a new way of "getting rich" with very low cost and considerable income. Every day, there are new loopholes and new systems on-line. People's inertia ensures that low-level problems come one after another. It's like giving money. What's worth mentioning here is that during this period, a very small security branch called "data leak Hunter" emerged, whose characteristics are: that is, the discovery and acquisition of leaked data without difference across the network, and the screening of important attributes of data, and the extraction of high-value target information. This is a model that costs nothing.

At first, the birth of data leak hunters was strongly related to Shodan. The technology can't be simpler: data acquisition for large databases (elastic search, Solr, Hadoop, mongodb, etc.) does not enable account authentication by default. The premise of great changes is that a search engine can tell you that there are a large number of such databases in the whole network, and can tell you the specific IP address. So, when a bunch of hackers run to Shodan to search the database and clear "take" the data, it's estimated that the author of Shodan is inexplicable. He didn't expect that the original intention was to analyze the networking hardware devices, resulting in a lot of people playing with the data. Compared with this, later hackers' prank to collect the whole network camera behavior that can "peep" (I Nsecam), it's just too small.

At this time, it has basically announced the arrival of an era. For those who have fantasized that "if I have loopholes, I may not be attacked, and the risk of my database leaking may not be equal to leaking", they are severely confronted with various security accidents. This is an era of data privacy disclosure for the whole people. Citizen information, bank card information, confidential unit information, this is not Something about a country is a global challenge, so the website has been launched. Everyone is a hacker, or everyone can be a hacker. What we see is just the tip of an extremely small iceberg, so that security personnel have lost confidence. Once a leader said that he had done a lot of technology to solve the privacy problem, and I said sadly, "is it still meaningful?" The leader objected, "no, we don't have another generation." I was very moved.

After telling these stories, I just want to tell you that when we eat, sleep and fall in love, there are a group of hackers who stare at the new loopholes and new online systems every day. They attack every server that can be controlled or utilized day and night, obtain sensitive data, destroy the system, and even use computing resources. As long as you have gaps, they will come in, which has nothing to do with whether you are an important unit or a high-value system. It's about economic interests, not technology. Some friends will say, steal it, destroy it, what's the relationship with me. Let's think about it a little bit. The data hackers steal include us, our bank card number, ID card number, social security number, home address, telephone number, our eating habits, and our business interests. What's more, one day there was a power failure and the subway was stopped. It was us who hurt. It's inconvenient for everyone to go out to build a road. Everyone will curse. If the traffic system is broken, it's no longer an economic problem. You will say that it's none of my business. It's a national business. Hey, just wait for your words: the ultimate buyer of network security must be the country.

When the U.S. upgraded cyber command to joint operations command, it officially declared the inevitability of cyber war. Otherwise, what kind of war should we do? It's just a country that invents computers, a country that invents networks, a country that Snowden has revealed all kinds of ways to destroy world peace and privacy by using networks, holding high the banner to say that in order to prevent enemies from damaging their national security. Well, I immediately remembered that when I used to identify the back door of a website, a very natural rule was that the first person who visited the back door address was the generator of the back door. There is a similar judgment. The first machine that is generated by the antivirus engine has a high probability of being the source of Trojans, because their first job is to test the ability to bypass the antivirus software. So let's protect ourselves first. When we come to discuss hackers, it is not petty theft, but the formal "army".

We live a good life in the physical world, and in the dark network forest, we can not assume that we are safe, but should ask what else is safe? There are a lot of security incidents that prove that tens of millions of devices on the Internet are controlled by hackers at every moment. With the arrival of the Internet of things, the Internet of vehicles, smart home, 5g, the era of the Internet of vehicles, as well as the official announcement and distribution of the IPv4 address, IPv6 has stepped onto the mainstream stage. In the coming era, we can expect that tens of billions of networking devices will formally access the network, and each product will experience the process from full leakage to gradual security. The vulnerability of an Internet of things device will easily let hundreds of thousands of Internet devices be controlled. This kind of attack is inevitable. You didn't do anything wrong, and I don't know who you are. The only problem is that you bought this device and let it connect with the Internet, "destroy you, what to do with you".

The more IP, the more complex the device types and the larger the data, the more a means is needed to quickly retrieve everything on the Internet. This means is the same as satellite guidance. On the one hand, it is always ready to attack everything. On the other hand, it is always ready to be attacked. The space defense that the space network is building is the same. You need to not only protect human operations, but also protect outer space. Look at the orbit of satellites, the route of meteorites, and even the invisible radio waves to see if they contain a series of mathematical symbols. This is the premise of establishing early warning mechanism.

Let's start with a little bit of hacking. If we were hackers, what would we do? Although I mentioned a lot of aimless attacks, I didn't say that high-value attacks are not fixed-point attacks. They are just waiting for the day when the city gate opens. So the hacker's behavior can be roughly divided into the following categories:

First, control network assets without fixed targets. Every day, they will focus on the output source of the whole network vulnerability (such as seclist, exploitdb, Metasploit, etc.), once there is POC available for control, they will write exp to directly do persistent control; data leakage also belongs to this range.

The second category is sensitive data stealing for specific business targets. The target is a commercial company with a specific scope, so the first thing they do is to establish the attack surface for the target. The process of vulnerability is similar to the first.

In the third category, sensitive data acquisition or direct destruction is carried out for specific basic units that affect the national economy and people's livelihood. This kind of attack should also be combed, but generally it is not one or two target units, but one industry or multiple industries. At the same time, the vulnerability is not obtained from the source, but more uses the 0day vulnerability that is self mined or purchased.

The above three types of hackers are not as clear-cut as we think. The so-called ideal is mostly the cover of pursuing profits. Their behavior is based on the market, so they will wander and shuttle among them.

For the third kind of protection, you must assume that the other party has 0 day, which is why unknown threat detection is one of the capabilities that must be possessed in the future. Nday can also end the fight, but 0day is more secretive. At first, they are all targeted attacks against specific targets, and then there are fan target attacks. It's OK for these people to make small noises. The high-value attack finally returns to the attack against specific targets. This essence has existed all the time, but from the previous hard junction to the present, they can take all of them and then determine whether they are important targets. The core of this process is two libraries: the vulnerability library updated every day and the attack surface of a specific target.

It's really not worth mentioning that vulnerability databases are very important. They are controlled by a few people. How to mine them is a science, how to collect them is also a science. It's difficult to automate them in a certain period of time. But the attack surface is different. It is also a science to comb the attack surface of a specific target, but this science can achieve certain automation relatively quickly. The comprehensiveness of attack surface combing will develop in two directions: one is the entities with specific targets open to the outside world, such as the most direct IP; the other is the components on the entities, such as middleware software system and other attributes. We can simply understand that one is macro, the other is micro, macro solves the width of the surface, and micro solves the depth of the description of a single entity. Here I put a picture of the difference in the eyes of defenders and attackers, which also determines the different ways of thinking.

In the eyes of the defending party, the attack surface, the accounting management, the hardware equipment and the IP system management are more important. In the eyes of the attacker, the attack surface focuses more on the business system, middleware, employees and all the vulnerable points that can be used in the periphery. So when the defense comes up with 10000 IPS, the hackers may see 100000 attack points.

Let's separate the macro and micro aspects mentioned above. First of all, we need to solve the problem of how many IP companies have access to the Internet. The simplest mode is to find the root domain name of the enterprise, then find all the subdomains, and then resolve all the subdomains to IP. It seems very simple, but it's good to find out 30% in this way. Who says there is only one root domain? Who says IP is bound to domain name? You have a few small questions: how to find all the root domains? How to find the IP address without domain name but with ownership? Hackers are smart. They say that we use the filing database to check, whois to check, certificates to check (this method is very useful), Section C to check, icon to check (this is already supported in Shodan, but it doesn't support domain names, otherwise it's more lethal). If we don't use keywords to check, hackers have a lot of methods, and some of them are patient, which will be sufficient Using existing tools. As a result, when hackers come in through all kinds of entrances, the defenders are shocked, all kinds of criticisms are pursued, and then they say, "who opened these IP addresses, why are they not in the monitoring range"!

To sum up, the macro development is that there will be more and more data. One is that there will be more and more IP collection (from IPv4 to IPv6), the other is that there will be more and more port and protocol support, and the other is that there will be more and more fields. How to solve the problem that all the IP and domain names using this SSL certificate can be reversely deduced through a certificate? First, collect all IP and domain names, and then analyze the certificate information deployed above. This workload is not difficult, it is the investment source. No enterprise wants to build a global library, so the asset management of the enterprise is top-down, and the method used by hackers is bottom-up. We should collect all of them first and then analyze them later. Let's give a few examples and feel it a little bit: Q =% Q = / at present, each has its own strengths, no one dominates the world.

Then the process of subsequent analysis is to go to the micro level. The same IP, the same port and protocol will correspond to different devices and applications, and of course, will correspond to different ownership subjects. So the micro development is to analyze as many attributes of this IP from the data as possible. For example, you can label it with ADSL, Internet of things, financial industry and street. These are the results of very typical fine-grained analysis, which correspond to the business requirements of different scenarios. For example, the database output size, number of records and other attributes can be used, and for example, screenshots can be provided for videos. We can think that the following five level classification chart represents the most basic business tag attribute (data chart comes from goby):

As can be seen from the figure above, we can analyze the hardware layer as VMware. The operating system layer above is CentOS, the service layer has nginx / MySQL / reidis / openssh, the support layer has struts 2, and the business layer has Jenkins / kibana system. The more detailed these micro data are portrayed, the faster the response speed will be in the future security emergency. We don't feel about an IP, but if there are millions of devices, if jeninks breaks a hole, then you can quickly screen out less than 100 devices from the millions of devices for emergency response. Some enterprises say that we have registration for these applications. In fact, there are many cases where the test environment set up by employees is not registered, and there are also cases where people are negligent. In this way, a gap is formed, and hackers will come in. The best way is to solve the problems caused by human beings in an automatic way. We solve the problems through technology and management. In fact, the time for hackers to complete the whole network attack from the vulnerability is no more than about an hour (we can simulate similar effects). Enterprises can imagine how we should deal with it.

For fine-grained in-depth analysis, we can use Shodan to slightly feel the output effect, such as database, which can describe database type, database quantity, data size, database list, database field, etc

What's the usage? What do you think is the use? Every database it can analyze can be downloaded. Any hacker can quickly complete the data stealing of the whole network, of course, they can also grasp the real-time information of global data threat, which is a high-level ability required by situation awareness.

Or look at the properties of video screenshots. Every open camera may be used to analyze the environment, whether it is a warehouse or a street, a school or a government department, a bedroom or an oil pipeline. What? Nude picture scandal? You use such a powerful weapon to do such a dirty work?

The finer and deeper the granularity, the higher the corresponding output value and the more specific the scenario. You can take a look at the original data, which contains more fine-grained content, such as labeling the cloud. Or labels for industrial controls:

There are too many contents involved, even including the historical vulnerability labels of CVE. In the next article, I will make a comparative analysis of the core differences between different platforms, focusing on them. The reality is much more complicated than you think. What you see is what I want you to see. What I don't want you to see is that you know nothing. At present, these are just the beginning, many scenes are constantly emerging, just like Google came out as a search database, then they did news aggregation, video, index, public opinion analysis What I am worried about is not the open all asset search engine. In this respect, Shodan, censys, fofa and zoomeye have their own advantages and disadvantages. It is difficult to have a subversive platform. However, for the search platform in a specific subdivision field, it is actually a point that everyone should pay attention to. For example, expand, which focuses on Internet of things devices, is a new opportunity to revive in the future.

Now everyone is talking about situational awareness, which I can't agree with. But I don't agree with some domestic practices of hype concept. The concept is changing with each passing day. Many security concepts have been put forward and developed from European and American countries, but they have been stepped out step by step. Every subdivision field has been completed by solid companies, and every subdivision field can live well. This is very obvious in Israel and Silicon Valley Several people can sell hundreds of millions, and knowledge is valuable. But the development in the first ten years in China is to choose the step-by-step development. There is no data and dare to follow it. It directly ignores the early stage of accumulation, but it does not mean that it is good. So relatively speaking, at present, the larger security companies in China are mostly comprehensive solution companies, and the security market in China used to be compliance market, so it is difficult to leave enough opportunities for the security capability team in the subdivision field.

I think you need to know your basic situation, whether it's situational awareness or overall network defense. You should always know how many attack areas there are, which is the basis of the foundation, and ensure that all attack areas are under the monitoring and protection system. Only in this way can you form correct cognition and make quick decisions. Obviously, important units in China still have a long way to go in this regard. Too many successful intrusions have proved that the attacked way is not that the firewall is useless, but that the way of arranging troops is wrong. I often use a metaphor: an enterprise uses an indestructible steel helmet to protect its head, no one can wear it, but you can hurt his heart, stomach, limbs, and blood runs through the inside, so the protection party has built a Macedonian defense line, and the enemy can do whatever he wants by bypassing the defense line.

Network security is a complex science with many branches. With the continuous change of IT technology, there are too many scenarios and corresponding technologies. For example, from the initial PC security, there are anti-virus and system security. Then there are network security, website security, database security, mobile security, wireless security, Internet of things security, cloud security, Big data security, Internet of vehicles security, 5g security. My conclusion is as follows: 1) no one can do all these things well. If there is, he must be an integrator. In the future, more and more opportunities will be provided for those who are capable of subdividing fields, because only in this way can we ensure the leading technology and the comprehensive defense; secondly, security will return to its essence, no matter how the technology develops, weapons will change into the same battlefield, and the premise of attack and defense is a local map. Only by unfolding this correct detailed map can we evolve from the brave blind man to the generals and generals who can really know ourselves and the enemy and win every battle. Cyberspace mapping is the way to provide maps.


Next notice: technical direction of different cyberspace mapping platforms. Comparison of advantages and disadvantages, future trend. I'll finish the next issue of life.

Next Trailer: death.