{"id":46931,"date":"2022-06-01T18:00:00","date_gmt":"2022-06-01T18:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/security\/blog\/?p=115350"},"modified":"2022-06-01T18:00:00","modified_gmt":"2022-06-01T18:00:00","slug":"using-python-to-unearth-a-goldmine-of-threat-intelligence-from-leaked-chat-logs","status":"publish","type":"post","link":"https:\/\/www.threatshub.org\/blog\/using-python-to-unearth-a-goldmine-of-threat-intelligence-from-leaked-chat-logs\/","title":{"rendered":"Using Python to unearth a goldmine of threat intelligence from leaked chat logs"},"content":{"rendered":"

Dealing with a great amount of data can be time consuming, thus using Python can be very powerful to help analysts sort information and extract the most relevant data for their investigation. The open-source tools library, MSTICPy<\/a>, for example, is a Python tool dedicated to threat intelligence. It aims to help threat analysts acquire, enrich, analyze, and visualize data.<\/p>\n

This blog provides a workflow for deeper data analysis and visualization using Python, as well as for extraction and analysis of indicators of compromise (IOCs) using MSTICPy. Data sets from the February 2022 leak of data from the ransomware-as-a-service (RaaS)<\/a> coordinated operation called \u201cConti\u201d is used as case study.<\/p>\n

An interactive Jupyter notebook<\/a> with related data is also available for analysts interested to do further data exploration.<\/p>\n

This research aims to provide a view into research methodology that may help other analysts apply Python to threat intelligence. Analysts can reuse the code and continue to explore the extracted information. Additionally, it offers an out-of-the-box methodology for analyzing chat logs, extracting IOCs, and improving threat intelligence and defense process using Python.<\/p>\n

Using Python to analyze the Conti network<\/h2>\n

On February 28, 2022, a Twitter account named @ContiLeaks (allegedly a Ukrainian researcher) began posting leaked Conti data on Twitter. The leaked data sets, which were posted in a span of several months, consisted of chat logs, source codes, and backend applications.<\/p>\n

For this research, we focused our analysis on the chat logs, which revealed crucial information about the Conti group\u2019s operating methods, infrastructure, and organizational structure.<\/p>\n

Compiling and translating chat logs<\/h3>\n

The leaked chat logs are written in the Russian language. To make the analysis more accessible, we adopted the methodology published here<\/a> and translated the logs to English.<\/p>\n

The chat logs revealed that the Conti group uses the messaging application Jabber to communicate among members. Since raw Jabber logs are saved using a file per day, they can be compiled in one JSON file so they can easily be manipulated with Python. Once the data is merged, they can be translated using the deep translator library. After the logs are translated and loaded into a new file, it\u2019s then possible to load the data into a dataframe for manipulation and exploration:<\/p>\n

df = pd.read_json(codecs.open('translated_Log2.json', 'r', 'utf-8'))<\/pre>\n
\"A
Figure 1. Translated logs<\/figcaption><\/figure>\n
Russian slang words not properly translated by the automated process can be translated by creating a dictionary. A dictionary off a list proposed here<\/a> was used in this case to correctly translate the slang:<\/p>\n
\"A
Figure 2. Translating slang<\/figcaption><\/figure>\n
Analyzing the chat activity timeline<\/h3>\n
One way to get insights from chat logs is to see its timeline and check the number of discussions per day. The Bokeh<\/a> library can be used to build an interactive diagram and explore the loaded dataframe.<\/p>\n
\"A
Figure 3. Python code for exploring discussions<\/figcaption><\/figure>\n
Using the data from Conti chat logs generates the following diagram, which shows the volume of Jabber discussions over time:<\/p>\n
\"A
Figure 4. Volume of discussions over time<\/figcaption><\/figure>\n
Visualizing the data as a timeline shows some peaks of activity that align to certain events. In the case of the Conti leaks, for example:<\/p>\n