peep everywhere

Posted by millikan at 2020-02-28

On July 31, 2013, Snowden, the whistleblower of the "Monitoring Gate" incident in the United States, broke the silence suddenly and broke the news again, exposing the details of the "xkeyscale" monitoring plan in the United States on a larger scale, which immediately attracted great attention.

Xkeyscore program background

The "xkeyscale" plan is a top secret monitoring project of the national security administration of the United States. It is the most extensive plan to steal network intelligence in 2013, covering almost all the behaviors of an ordinary user on the Internet, including email content, network access and search, and related metadata. To support the project, the national security agency has more than 700 servers in 150 locations around the world. Agents only need to input simple information such as the object's email to monitor it in real time. Snowden said he had the opportunity to use the "xkeyscale" program when he was employed by the National Security Agency (NSA). He once described that as long as there is a corresponding e-mail address, he can monitor anyone, from civilians to judges and presidents.

The "xkeyscale" program has helped us intelligence agencies capture hundreds of terror suspects, but there are still concerns about such a large-scale monitoring program. The monitoring program called "xkeyscale" can cover almost all online information and collect Internet data to the largest extent, including e-mail, website information, search and chat records, etc. US intelligence analysts can even "real-time monitor" individual Internet activities through the "xkeyscale" program. In 2012, "xkeyscale" stored up to 41 billion monitoring data records in one month.

NSA boasted in the presentation that the program helped the United States capture 300 terrorists by 2008. The program can monitor any American technically, and analysts can get relevant data even without approval.  

NSA has a tool called 'DNI PRESENTER' that can be used to read the mail content that users keep. NSA analysts using "xkeyscale" can also use the tool to read chat records or privacy information of Facebook users. NSA analysts can monitor Facebook users' chat records simply by typing their Facebook user name and setting a time period for finding relevant information.

The arrest of Brandon Raub, a former Marine of the United States, proves that the xkeyscale program was used. He was arrested in August 2012 and subsequently sent to a mental hospital. The reason for Laub's arrest is that according to what he said on Facebook, he is believed to have a "terrorist nature.". In an interview with Sai, John Whitehead, Laub's principal lawyer, said that the "terrorist nature" content of Laub's post on Facebook was actually part of the one Laub was playing with About Facebook's personal games, Laub is playing the game called Illuminati with his brother and sister, while the U.S. government has checked the chat records of Laub and others.

Analysis on the framework of "xkeyscale" plan

The xkeyscale program claims to monitor more than 700 servers in nearly 150 locations. The monitoring server distribution is shown in Figure 1. The project can enable intelligence personnel to "real-time monitor" individual Internet activities without prior authorization, and even extract Chinese information.

Figure 1 Distribution of servers monitored by xkeyscale

The system can obtain data from the tailed access operations (TAO) to show analysts that all devices with vulnerabilities can be exploited in a certain country.

Figure 2 system function diagram

As shown in Figure 2, "xkeyscale" can load Tao data, display available electronic devices, and analysts can "store any information they want to extract". On a slide entitled "entity extraction", the ability to extract information in English, Arabic and Chinese was also highlighted.

Figure 3 entity extraction focuses on Arabic and Chinese besides English

As shown in Figure 3, the project "can cover almost all the online behaviors of a netizen", including the content of e-mail, websites visited, search and chat records, etc. Using "xkeyscale" and other surveillance programs, intelligence analysts can "real-time monitor" an individual's Internet activities just by knowing someone's email address or IP address.

The project allows analysts to search the metadata and content of online activities. In addition to e-mail and IP address, they can also search by name, phone, keywords, etc. The document said that "strong selection (searching by email) only provides a very limited ability, because a large number of online behaviors are anonymous".

Figure 4 search level in the background of xkeyscale

Because searching content in a complete database produces too many results, analysts search metadata first. As shown in Figure 4, the "xkeyscale" background adopts hierarchical search. In 2012, "xkeyscale" stored up to 41 billion monitoring data records in one month. Because the monitoring data generated by "xkeyscale" is too large, the content can only be stored in the system for 3 to 5 days, and the metadata can be stored for 1 month. NSA has thus created a new database where analysts can "save" their "interesting" intelligence for up to five years.

Analysts can search by name and email domain name. The system will search e-mail, web pages and documents for the content after "send to, receive from, CC and contact us" page, as shown in Figure 5.

Figure 5 xkeyscale email search tutorial

But "xkeyscale" will list all contents judged as e-mail, which is easy to mislead analysts, as shown in Figure 6.

Figure 6. Xkeyscale automatically determines which content is email information

U.S. law requires that there must be a corresponding authorization when monitoring Americans, but the technology of "xkeyscale" can completely break this limitation. Moreover, analysts can designate the target as "foreigner" through very simple operation. In a drop-down menu, the system lists a variety of reasons, such as "the target claims to be outside the United States", "field intelligence personnel, foreign governments point out that the target is outside the country", "the storage media used by the target is outside the country", "the phone number is registered as a foreign country", "open information" and "network, hardware and other technical information show that the target is outside the country" The analyst only needs to check one item, and all subsequent operations are "legal", as shown in Figure 7.

Figure 7. Xkeyscale allows analysts to "one click" who is a "foreigner"

The most commonly used retrieval method of this system is to search e-mail, but its retrieval ability is far from limited to this. It can even read chat messages on social networking sites. Analysts only need to type in the target (such as a user name in Facebook) in NSA's DNI presenter tool, and then they can read the user's interaction information on Facebook through the "xkeyscale" plan, as shown in Figure 8.

Figure 8 xkeyscale can monitor Facebook chat and private messages

"Xkeyscale" can also monitor which visitors a specific website has, as shown in Figure 9.

Figure 9 xkeyscale can monitor the website to track down visitors


In terms of various performances, "xkeyscale" has a wide range of ideas, and its information collection and analysis capabilities are astonishing and chilling. It is like a pair of omnipresent eyes, analyzing and perceiving our daily work and life. In order to block these eyes, we should not only improve our awareness of security and confidentiality, but also strengthen the technology research of privacy protection and use newer and better technology to resist attacks.

Science and technology branch of China confidentiality Association

Please long press QR code to identify and pay attention to the micro signal of science and technology branch of China Association for confidentiality.