Collect All Functions


This project comes with functions that collect all STIX objects from all ATT&CK Matrices at once. These functions help collect more with less API call requests.

Import ATTACK API Client

from attackcti import attack_client

Import Extra Libraries

from pandas import *
import json

import logging
logging.getLogger('taxii2client').setLevel(logging.CRITICAL)
pandas.__version__
'1.3.5'

Initialize ATT&CK Client Variable

lift = attack_client()

Get All Techniques

We can extract all STIX objects of type attack-pattern (technique) across all ATT&CK matrices.

  • By default, this function removes deprecated and revoked techniques. If you want to keep those techniques in the results, you can run the function with the parameter skip_revoked_deprecated=False.

  • By default, this function also includes all techniques and sub-techniques. If you want to only get techniques that are not sub-techniques, you can run the function with the parameter include_subtechniques=False.

techniques = lift.get_techniques()
print("Number of Techniques in ATT&CK")
len(techniques)
Number of Techniques in ATT&CK
736

By default, the data returned by the available functions in the attackcti library is of type stix2. However, if you want to interact with libraries such as Pandas, it needs to be of type dict

all_techniques = []
for t in techniques:
    all_techniques.append(json.loads(t.serialize()))
df = pandas.json_normalize(all_techniques)
df.reindex(['created','name', 'x_mitre_data_sources', 'x_mitre_platforms'], axis=1)[0:5]
created name x_mitre_data_sources x_mitre_platforms
0 2021-10-12T20:02:31.866Z Resource Forking [File: File Creation, Process: Process Creatio... [macOS]
1 2021-10-08T14:06:28.212Z Downgrade Attack [Command: Command Execution, Process: Process ... [Windows, Linux, macOS]
2 2021-10-05T21:26:15.081Z Login Items [Process: Process Creation, File: File Modific... [macOS]
3 2021-10-05T01:15:06.293Z Reflective Code Loading [Script: Script Execution, Process: OS API Exe... [macOS, Linux, Windows]
4 2021-10-01T17:58:26.445Z Cloud Storage Object Discovery [Cloud Storage: Cloud Storage Enumeration, Clo... [IaaS]

We can now access the schema of the dataframe

list(df)
['type',
 'id',
 'created_by_ref',
 'created',
 'modified',
 'name',
 'description',
 'kill_chain_phases',
 'external_references',
 'object_marking_refs',
 'x_mitre_contributors',
 'x_mitre_data_sources',
 'x_mitre_defense_bypassed',
 'x_mitre_detection',
 'x_mitre_is_subtechnique',
 'x_mitre_permissions_required',
 'x_mitre_platforms',
 'x_mitre_version',
 'x_mitre_remote_support',
 'x_mitre_system_requirements',
 'x_mitre_network_requirements',
 'x_mitre_effective_permissions',
 'x_mitre_impact_type',
 'x_mitre_tactic_type',
 'x_mitre_old_attack_id']

Showing one technique example:

techniques[0]
AttackPattern(type='attack-pattern', id='attack-pattern--b22e5153-ac28-4cc6-865c-2054e36285cb', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2021-10-12T20:02:31.866Z', modified='2021-10-16T01:50:40.276Z', name='Resource Forking', description='Adversaries may abuse resource forks to hide malicious code or executables to evade detection and bypass security applications. A resource fork provides applications a structured way to store resources such as thumbnail images, menu definitions, icons, dialog boxes, and code.(Citation: macOS Hierarchical File System Overview) Usage of a resource fork is identifiable when displaying a file’s extended attributes, using <code>ls -l@</code> or <code>xattr -l</code> commands. Resource forks have been deprecated and replaced with the application bundle structure. Non-localized resources are placed at the top level directory of an application bundle, while localized resources are placed in the <code>/Resources</code> folder.(Citation: Resource and Data Forks)(Citation: ELC Extended Attributes)\n\nAdversaries can use resource forks to hide malicious data that may otherwise be stored directly in files. Adversaries can execute content with an attached resource fork, at a specified offset, that is moved to an executable location then invoked. Resource fork content may also be obfuscated/encrypted until execution.(Citation: sentinellabs resource named fork 2020)(Citation: tau bundlore erika noerenberg 2020)', kill_chain_phases=[KillChainPhase(kill_chain_name='mitre-attack', phase_name='defense-evasion')], revoked=False, external_references=[ExternalReference(source_name='mitre-attack', url='https://attack.mitre.org/techniques/T1564/009', external_id='T1564.009'), ExternalReference(source_name='macOS Hierarchical File System Overview', description='Tenon. (n.d.). Retrieved October 12, 2021.', url='http://tenon.com/products/codebuilder/User_Guide/6_File_Systems.html#anchor520553'), ExternalReference(source_name='Resource and Data Forks', description='Flylib. (n.d.). Identifying Resource and Data Forks. Retrieved October 12, 2021.', url='https://flylib.com/books/en/4.395.1.192/1/'), ExternalReference(source_name='ELC Extended Attributes', description="Howard Oakley. (2020, October 24). There's more to files than data: Extended Attributes. Retrieved October 12, 2021.", url='https://eclecticlight.co/2020/10/24/theres-more-to-files-than-data-extended-attributes/'), ExternalReference(source_name='sentinellabs resource named fork 2020', description='Phil Stokes. (2020, November 5). Resourceful macOS Malware Hides in Named Fork. Retrieved October 12, 2021.', url='https://www.sentinelone.com/labs/resourceful-macos-malware-hides-in-named-fork/'), ExternalReference(source_name='tau bundlore erika noerenberg 2020', description='Erika Noerenberg. (2020, June 29). TAU Threat Analysis: Bundlore (macOS) mm-install-macos. Retrieved October 12, 2021.', url='https://blogs.vmware.com/security/2020/06/tau-threat-analysis-bundlore-macos-mm-install-macos.html')], object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'], x_mitre_contributors=['Jaron Bradley @jbradley89', 'Ivan Sinyakov'], x_mitre_data_sources=['File: File Creation', 'Process: Process Creation', 'File: File Metadata', 'Command: Command Execution'], x_mitre_defense_bypassed=['Notarization; Gatekeeper'], x_mitre_detection='Identify files with the <code>com.apple.ResourceFork</code> extended attribute and large data amounts stored in resource forks. \n\nMonitor command-line activity leveraging the use of resource forks, especially those immediately followed by potentially malicious activity such as creating network connections. ', x_mitre_is_subtechnique=True, x_mitre_permissions_required=['User'], x_mitre_platforms=['macOS'], x_mitre_version='1.0')

Enrich Techniques Data Sources

As you might already know, the ATT&CK data model now represents data sources as objects. However, when retrieving techniques from ATT&CK TAXII server, their data sources section only includes data sources and data components names. Therefore, we created a parameter that you can use with the get_techniques() function to enrich the data sources section of each technique. The parameter enrich_data_sources is set to False by default.

techniques = lift.get_techniques(enrich_data_sources=True)

This function returns a list of techniques with the x_mitre_data_sources attribute as a list of STIX objects representing data sources with their respective data components depending on the technique’s detection context.

for ds in techniques[0]['x_mitre_data_sources']:
    for dc in ds['data_components']:
        print(ds['name'], '-', dc['name'])
Command - Command Execution
File - File Metadata
File - File Creation
Process - Process Creation
techniques[0]['x_mitre_data_sources'][0]
{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],
 'modified': '2021-11-10T09:30:48.694Z',
 'name': 'Command',
 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',
 'type': 'x-mitre-data-source',
 'id': 'x-mitre-data-source--73691708-ffb5-4e29-906d-f485f6fa7089',
 'description': 'A directive given to a computer program, acting as an interpreter of some kind, in order to perform a specific task(Citation: Confluence Linux Command  Line)(Citation: Audit OSX)',
 'created': '2021-10-20T15:05:19.273Z',
 'external_references': [{'url': 'https://attack.mitre.org/datasources/DS0017',
   'external_id': 'DS0017',
   'source_name': 'mitre-attack'},
  {'url': 'https://confluence.atlassian.com/confkb/how-to-enable-command-line-audit-logging-in-linux-956166545.html',
   'description': 'Confluence Support. (2021, September 8). How to enable command line audit logging in linux. Retrieved September 23, 2021.',
   'source_name': 'Confluence Linux Command  Line'},
  {'url': 'https://www.scip.ch/en/?labs.20150108',
   'description': 'Gagliardi, R. (n.d.). Audit in a OS X System. Retrieved September 23, 2021.',
   'source_name': 'Audit OSX'}],
 'x_mitre_version': '1.0',
 'x_mitre_platforms': ['Windows', 'Linux', 'macOS', 'Network', 'Containers'],
 'x_mitre_collection_layers': ['Host', 'Container'],
 'x_mitre_contributors': ['Austin Clark',
  'Center for Threat-Informed Defense (CTID)'],
 'data_components': [{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],
   'modified': '2021-10-20T15:05:19.273Z',
   'id': 'x-mitre-data-component--685f917a-e95e-4ba0-ade1-c7d354dae6e0',
   'description': 'Invoking a computer program directive to perform a specific task (ex: Windows EID 4688 of cmd.exe showing command-line parameters, ~/.bash_history, or ~/.zsh_history)',
   'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',
   'name': 'Command Execution',
   'created': '2021-10-20T15:05:19.273Z',
   'type': 'x-mitre-data-component',
   'x_mitre_version': '1.0',
   'x_mitre_data_source_ref': 'x-mitre-data-source--73691708-ffb5-4e29-906d-f485f6fa7089'}]}

Get All Groups

We can also extract all the available groups across all ATT&CK matrices at once.

groups = lift.get_groups()
print("Number of Groups in ATT&CK")
len(groups)
Number of Groups in ATT&CK
131
groups_list = []
for t in groups:
    groups_list.append(json.loads(t.serialize()))
df = pandas.json_normalize(groups_list)
df[0:4]
type id created_by_ref created modified name description aliases external_references object_marking_refs x_mitre_contributors x_mitre_version
0 intrusion-set intrusion-set--35d1b3be-49d4-42f1-aaa6-ef159c8... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-10-01T01:57:31.229Z 2021-10-15T18:47:18.824Z TeamTNT [TeamTNT](https://attack.mitre.org/groups/G013... [TeamTNT] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [Will Thomas, Cyjax] 1.0
1 intrusion-set intrusion-set--39d6890e-7f23-4474-b8ef-e7b0343... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-29T15:10:19.236Z 2021-10-15T15:16:47.329Z Andariel [Andariel](https://attack.mitre.org/groups/G01... [Andariel, Silent Chollima] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [Kyoung-ju Kwak (S2W)] 1.0
2 intrusion-set intrusion-set--6566aac9-dad8-4332-ae73-20c23ba... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-28T17:41:12.950Z 2021-10-25T14:28:10.337Z Ferocious Kitten [Ferocious Kitten](https://attack.mitre.org/gr... [Ferocious Kitten] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [Pooja Natarajan, NEC Corporation India, Manik... 1.0
3 intrusion-set intrusion-set--e5603ea8-4c36-40e7-b7af-a077d24... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-24T21:41:34.797Z 2021-10-16T02:06:06.404Z IndigoZebra [IndigoZebra](https://attack.mitre.org/groups/... [IndigoZebra] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [Pooja Natarajan, NEC Corporation India, Yoshi... 1.0

Get All Software

We can extract all Enterprise, Mobile and ICS (Software Malware & Tools)

software = lift.get_software()
print("Number of Software in ATT&CK")
len(software)
Number of Software in ATT&CK
641
software_list = []
for t in software:
    software_list.append(json.loads(t.serialize()))
df = pandas.json_normalize(software_list)
df[0:4]
type id created_by_ref created modified name description labels external_references object_marking_refs x_mitre_aliases x_mitre_contributors x_mitre_platforms x_mitre_version x_mitre_old_attack_id
0 tool tool--f91162cc-1686-4ff8-8115-bf3f61a4cc7a identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-14T21:45:30.280Z 2021-09-21T18:03:13.205Z Wevtutil [Wevtutil](https://attack.mitre.org/software/S... [tool] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [Wevtutil] [Viren Chaudhari, Qualys, Harshal Tupsamudre, ... [Windows] 1.0 NaN
1 tool tool--11f8d7eb-1927-4806-9267-3a11d4d4d6be identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-07-30T15:43:17.770Z 2021-10-15T15:49:25.284Z Sliver [Sliver](https://attack.mitre.org/software/S06... [tool] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [Sliver] [Achute Sharma, Keysight, Ayan Saha, Keysight] [Windows, Linux, macOS] 1.0 NaN
2 tool tool--80c815bb-b24a-4b9c-9d73-ff4c075a278d identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-03-19T13:11:50.666Z 2021-04-26T22:35:19.315Z Out1 [Out1](https://attack.mitre.org/software/S0594... [tool] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [Out1] NaN [Windows] 1.0 NaN
3 tool tool--03c6e0ea-96d3-4b23-9afb-05055663cf4b identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-03-18T14:57:34.628Z 2021-04-25T23:30:38.375Z RemoteUtilities [RemoteUtilities](https://attack.mitre.org/sof... [tool] [{'source_name': 'mitre-attack', 'url': 'https... [marking-definition--fa42a846-8d90-4e51-bc29-7... [RemoteUtilities] NaN [Windows] 1.0 NaN

Get All Relationships

We can also get all relationships from all ATT&CK matrices with one API request.

relationships = lift.get_relationships()
print("Number of Relationships in ATT&CK")
len(relationships)
Number of Relationships in ATT&CK
15752
relations_list = []
for t in relationships:
    relations_list.append(json.loads(t.serialize()))
df = pandas.json_normalize(relations_list)
df[0:4]
type id created_by_ref created modified relationship_type source_ref target_ref object_marking_refs description external_references
0 relationship relationship--9567076b-2a77-43e4-befd-19556def... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z 2021-11-10T09:30:48.753Z detects x-mitre-data-component--3d20385b-24ef-40e1-9f5... attack-pattern--910906dd-8c0a-475a-9cc1-5e029e... [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN NaN
1 relationship relationship--79fa693d-38b2-4730-8602-1f72eef5... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z 2021-11-10T09:30:48.753Z detects x-mitre-data-component--9ce98c86-8d30-4043-ba5... attack-pattern--60d0c01d-e2bf-49dd-a453-f8a9c9... [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN NaN
2 relationship relationship--ed1c4fff-998f-499d-8a00-cfdee554... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z 2021-11-10T09:30:48.753Z detects x-mitre-data-component--9bde2f9d-a695-4344-bfa... attack-pattern--2959d63f-73fd-46a1-abd2-109d7d... [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN NaN
3 relationship relationship--41c0352d-b377-4fe9-8c3a-67b78a9a... identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z 2021-11-10T09:30:48.753Z detects x-mitre-data-component--c0a4a086-cc20-4e1e-b7c... attack-pattern--6836813e-8ec8-4375-b459-abb388... [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN NaN

Get All Data Sources

Now that data sources are part of the ATT&CK data model as objects, we can retrieve all that information at once.

data_sources = lift.get_data_sources()
print("Number of Data Sources in ATT&CK")
len(data_sources)
Number of Data Sources in ATT&CK
38
data_sources[0]
{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],
 'modified': '2021-10-20T15:05:19.275Z',
 'name': 'Internet Scan',
 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',
 'type': 'x-mitre-data-source',
 'id': 'x-mitre-data-source--38fe306c-bdec-4f3d-8521-b72dd32dbd17',
 'description': 'Information obtained (commonly via active network traffic probes or web crawling) regarding various types of resources and servers connected to the public Internet',
 'created': '2021-10-20T15:05:19.275Z',
 'external_references': [{'url': 'https://attack.mitre.org/datasources/DS0035',
   'external_id': 'DS0035',
   'source_name': 'mitre-attack'}],
 'x_mitre_version': '1.0',
 'x_mitre_platforms': ['PRE'],
 'x_mitre_collection_layers': ['OSINT'],
 'x_mitre_contributors': []}

Get All Data Components

Now that data components are also part of the ATT&CK data model as objects, we can retrieve all that information at once.

data_components = lift.get_data_components()
print("Number of data components in ATT&CK")
len(data_components)
Number of data components in ATT&CK
109
data_components[0]
{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],
 'modified': '2021-10-20T15:05:19.275Z',
 'id': 'x-mitre-data-component--cc150ad8-ecfa-4340-9aaa-d21165873bd4',
 'description': 'Logged domain name system (DNS) data highlighting timelines of domain to IP address resolutions (ex: passive DNS)',
 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',
 'name': 'Passive DNS',
 'created': '2021-10-20T15:05:19.275Z',
 'type': 'x-mitre-data-component',
 'x_mitre_version': '1.0',
 'x_mitre_data_source_ref': 'x-mitre-data-source--dd75f457-8dc0-4a24-9ae5-4b61c33af866'}