{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Collect All Functions\n", "------------------\n", "This project comes with functions that collect all STIX objects from all ATT&CK Matrices at once. These functions help collect more with less API call requests." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import ATTACK API Client" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from attackcti import attack_client" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import Extra Libraries" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from pandas import *\n", "import json\n", "\n", "import logging\n", "logging.getLogger('taxii2client').setLevel(logging.CRITICAL)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'1.3.5'" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pandas.__version__" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialize ATT&CK Client Variable" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "lift = attack_client()" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Get All Techniques\n", "We can extract all STIX objects of type `attack-pattern` (technique) across all ATT&CK matrices.\n", "* By default, this function removes `deprecated` and `revoked` techniques. If you want to keep those techniques in the results, you can run the function with the parameter `skip_revoked_deprecated=False`.\n", "* By default, this function also includes all techniques and sub-techniques. If you want to only get techniques that are not sub-techniques, you can run the function with the parameter `include_subtechniques=False`." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "techniques = lift.get_techniques()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of Techniques in ATT&CK\n" ] }, { "data": { "text/plain": [ "736" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(\"Number of Techniques in ATT&CK\")\n", "len(techniques)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By default, the data returned by the available functions in the attackcti library is of type **stix2**. However, if you want to interact with libraries such as **Pandas**, it needs to be of type **dict**" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
creatednamex_mitre_data_sourcesx_mitre_platforms
02021-10-12T20:02:31.866ZResource Forking[File: File Creation, Process: Process Creatio...[macOS]
12021-10-08T14:06:28.212ZDowngrade Attack[Command: Command Execution, Process: Process ...[Windows, Linux, macOS]
22021-10-05T21:26:15.081ZLogin Items[Process: Process Creation, File: File Modific...[macOS]
32021-10-05T01:15:06.293ZReflective Code Loading[Script: Script Execution, Process: OS API Exe...[macOS, Linux, Windows]
42021-10-01T17:58:26.445ZCloud Storage Object Discovery[Cloud Storage: Cloud Storage Enumeration, Clo...[IaaS]
\n", "
" ], "text/plain": [ " created name \\\n", "0 2021-10-12T20:02:31.866Z Resource Forking \n", "1 2021-10-08T14:06:28.212Z Downgrade Attack \n", "2 2021-10-05T21:26:15.081Z Login Items \n", "3 2021-10-05T01:15:06.293Z Reflective Code Loading \n", "4 2021-10-01T17:58:26.445Z Cloud Storage Object Discovery \n", "\n", " x_mitre_data_sources x_mitre_platforms \n", "0 [File: File Creation, Process: Process Creatio... [macOS] \n", "1 [Command: Command Execution, Process: Process ... [Windows, Linux, macOS] \n", "2 [Process: Process Creation, File: File Modific... [macOS] \n", "3 [Script: Script Execution, Process: OS API Exe... [macOS, Linux, Windows] \n", "4 [Cloud Storage: Cloud Storage Enumeration, Clo... [IaaS] " ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "all_techniques = []\n", "for t in techniques:\n", " all_techniques.append(json.loads(t.serialize()))\n", "df = pandas.json_normalize(all_techniques)\n", "df.reindex(['created','name', 'x_mitre_data_sources', 'x_mitre_platforms'], axis=1)[0:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now access the schema of the dataframe" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['type',\n", " 'id',\n", " 'created_by_ref',\n", " 'created',\n", " 'modified',\n", " 'name',\n", " 'description',\n", " 'kill_chain_phases',\n", " 'external_references',\n", " 'object_marking_refs',\n", " 'x_mitre_contributors',\n", " 'x_mitre_data_sources',\n", " 'x_mitre_defense_bypassed',\n", " 'x_mitre_detection',\n", " 'x_mitre_is_subtechnique',\n", " 'x_mitre_permissions_required',\n", " 'x_mitre_platforms',\n", " 'x_mitre_version',\n", " 'x_mitre_remote_support',\n", " 'x_mitre_system_requirements',\n", " 'x_mitre_network_requirements',\n", " 'x_mitre_effective_permissions',\n", " 'x_mitre_impact_type',\n", " 'x_mitre_tactic_type',\n", " 'x_mitre_old_attack_id']" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Showing one technique example:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "AttackPattern(type='attack-pattern', id='attack-pattern--b22e5153-ac28-4cc6-865c-2054e36285cb', created_by_ref='identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5', created='2021-10-12T20:02:31.866Z', modified='2021-10-16T01:50:40.276Z', name='Resource Forking', description='Adversaries may abuse resource forks to hide malicious code or executables to evade detection and bypass security applications. A resource fork provides applications a structured way to store resources such as thumbnail images, menu definitions, icons, dialog boxes, and code.(Citation: macOS Hierarchical File System Overview) Usage of a resource fork is identifiable when displaying a file’s extended attributes, using ls -l@ or xattr -l commands. Resource forks have been deprecated and replaced with the application bundle structure. Non-localized resources are placed at the top level directory of an application bundle, while localized resources are placed in the /Resources folder.(Citation: Resource and Data Forks)(Citation: ELC Extended Attributes)\\n\\nAdversaries can use resource forks to hide malicious data that may otherwise be stored directly in files. Adversaries can execute content with an attached resource fork, at a specified offset, that is moved to an executable location then invoked. Resource fork content may also be obfuscated/encrypted until execution.(Citation: sentinellabs resource named fork 2020)(Citation: tau bundlore erika noerenberg 2020)', kill_chain_phases=[KillChainPhase(kill_chain_name='mitre-attack', phase_name='defense-evasion')], revoked=False, external_references=[ExternalReference(source_name='mitre-attack', url='https://attack.mitre.org/techniques/T1564/009', external_id='T1564.009'), ExternalReference(source_name='macOS Hierarchical File System Overview', description='Tenon. (n.d.). Retrieved October 12, 2021.', url='http://tenon.com/products/codebuilder/User_Guide/6_File_Systems.html#anchor520553'), ExternalReference(source_name='Resource and Data Forks', description='Flylib. (n.d.). Identifying Resource and Data Forks. Retrieved October 12, 2021.', url='https://flylib.com/books/en/4.395.1.192/1/'), ExternalReference(source_name='ELC Extended Attributes', description=\"Howard Oakley. (2020, October 24). There's more to files than data: Extended Attributes. Retrieved October 12, 2021.\", url='https://eclecticlight.co/2020/10/24/theres-more-to-files-than-data-extended-attributes/'), ExternalReference(source_name='sentinellabs resource named fork 2020', description='Phil Stokes. (2020, November 5). Resourceful macOS Malware Hides in Named Fork. Retrieved October 12, 2021.', url='https://www.sentinelone.com/labs/resourceful-macos-malware-hides-in-named-fork/'), ExternalReference(source_name='tau bundlore erika noerenberg 2020', description='Erika Noerenberg. (2020, June 29). TAU Threat Analysis: Bundlore (macOS) mm-install-macos. Retrieved October 12, 2021.', url='https://blogs.vmware.com/security/2020/06/tau-threat-analysis-bundlore-macos-mm-install-macos.html')], object_marking_refs=['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'], x_mitre_contributors=['Jaron Bradley @jbradley89', 'Ivan Sinyakov'], x_mitre_data_sources=['File: File Creation', 'Process: Process Creation', 'File: File Metadata', 'Command: Command Execution'], x_mitre_defense_bypassed=['Notarization; Gatekeeper'], x_mitre_detection='Identify files with the com.apple.ResourceFork extended attribute and large data amounts stored in resource forks. \\n\\nMonitor command-line activity leveraging the use of resource forks, especially those immediately followed by potentially malicious activity such as creating network connections. ', x_mitre_is_subtechnique=True, x_mitre_permissions_required=['User'], x_mitre_platforms=['macOS'], x_mitre_version='1.0')" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "techniques[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Enrich Techniques Data Sources\n", "As you might already know, the ATT&CK data model now represents `data sources` as objects. However, when retrieving techniques from ATT&CK TAXII server, their data sources section only includes data sources and data components names. Therefore, we created a parameter that you can use with the `get_techniques()` function to enrich the data sources section of each technique. The parameter `enrich_data_sources` is set to `False` by default." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "techniques = lift.get_techniques(enrich_data_sources=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This function returns a list of techniques with the `x_mitre_data_sources` attribute as a list of STIX objects representing data sources with their respective data components depending on the technique's detection context." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Command - Command Execution\n", "File - File Metadata\n", "File - File Creation\n", "Process - Process Creation\n" ] } ], "source": [ "for ds in techniques[0]['x_mitre_data_sources']:\n", " for dc in ds['data_components']:\n", " print(ds['name'], '-', dc['name'])" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],\n", " 'modified': '2021-11-10T09:30:48.694Z',\n", " 'name': 'Command',\n", " 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',\n", " 'type': 'x-mitre-data-source',\n", " 'id': 'x-mitre-data-source--73691708-ffb5-4e29-906d-f485f6fa7089',\n", " 'description': 'A directive given to a computer program, acting as an interpreter of some kind, in order to perform a specific task(Citation: Confluence Linux Command Line)(Citation: Audit OSX)',\n", " 'created': '2021-10-20T15:05:19.273Z',\n", " 'external_references': [{'url': 'https://attack.mitre.org/datasources/DS0017',\n", " 'external_id': 'DS0017',\n", " 'source_name': 'mitre-attack'},\n", " {'url': 'https://confluence.atlassian.com/confkb/how-to-enable-command-line-audit-logging-in-linux-956166545.html',\n", " 'description': 'Confluence Support. (2021, September 8). How to enable command line audit logging in linux. Retrieved September 23, 2021.',\n", " 'source_name': 'Confluence Linux Command Line'},\n", " {'url': 'https://www.scip.ch/en/?labs.20150108',\n", " 'description': 'Gagliardi, R. (n.d.). Audit in a OS X System. Retrieved September 23, 2021.',\n", " 'source_name': 'Audit OSX'}],\n", " 'x_mitre_version': '1.0',\n", " 'x_mitre_platforms': ['Windows', 'Linux', 'macOS', 'Network', 'Containers'],\n", " 'x_mitre_collection_layers': ['Host', 'Container'],\n", " 'x_mitre_contributors': ['Austin Clark',\n", " 'Center for Threat-Informed Defense (CTID)'],\n", " 'data_components': [{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],\n", " 'modified': '2021-10-20T15:05:19.273Z',\n", " 'id': 'x-mitre-data-component--685f917a-e95e-4ba0-ade1-c7d354dae6e0',\n", " 'description': 'Invoking a computer program directive to perform a specific task (ex: Windows EID 4688 of cmd.exe showing command-line parameters, ~/.bash_history, or ~/.zsh_history)',\n", " 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',\n", " 'name': 'Command Execution',\n", " 'created': '2021-10-20T15:05:19.273Z',\n", " 'type': 'x-mitre-data-component',\n", " 'x_mitre_version': '1.0',\n", " 'x_mitre_data_source_ref': 'x-mitre-data-source--73691708-ffb5-4e29-906d-f485f6fa7089'}]}" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "techniques[0]['x_mitre_data_sources'][0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get All Groups\n", "We can also extract all the available groups across all ATT&CK matrices at once." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "groups = lift.get_groups()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of Groups in ATT&CK\n", "131\n" ] } ], "source": [ "print(\"Number of Groups in ATT&CK\")\n", "len(groups)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
typeidcreated_by_refcreatedmodifiednamedescriptionaliasesexternal_referencesobject_marking_refsx_mitre_contributorsx_mitre_version
0intrusion-setintrusion-set--35d1b3be-49d4-42f1-aaa6-ef159c8...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-10-01T01:57:31.229Z2021-10-15T18:47:18.824ZTeamTNT[TeamTNT](https://attack.mitre.org/groups/G013...[TeamTNT][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[Will Thomas, Cyjax]1.0
1intrusion-setintrusion-set--39d6890e-7f23-4474-b8ef-e7b0343...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-09-29T15:10:19.236Z2021-10-15T15:16:47.329ZAndariel[Andariel](https://attack.mitre.org/groups/G01...[Andariel, Silent Chollima][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[Kyoung-ju Kwak (S2W)]1.0
2intrusion-setintrusion-set--6566aac9-dad8-4332-ae73-20c23ba...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-09-28T17:41:12.950Z2021-10-25T14:28:10.337ZFerocious Kitten[Ferocious Kitten](https://attack.mitre.org/gr...[Ferocious Kitten][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[Pooja Natarajan, NEC Corporation India, Manik...1.0
3intrusion-setintrusion-set--e5603ea8-4c36-40e7-b7af-a077d24...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-09-24T21:41:34.797Z2021-10-16T02:06:06.404ZIndigoZebra[IndigoZebra](https://attack.mitre.org/groups/...[IndigoZebra][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[Pooja Natarajan, NEC Corporation India, Yoshi...1.0
\n", "
" ], "text/plain": [ " type id \\\n", "0 intrusion-set intrusion-set--35d1b3be-49d4-42f1-aaa6-ef159c8... \n", "1 intrusion-set intrusion-set--39d6890e-7f23-4474-b8ef-e7b0343... \n", "2 intrusion-set intrusion-set--6566aac9-dad8-4332-ae73-20c23ba... \n", "3 intrusion-set intrusion-set--e5603ea8-4c36-40e7-b7af-a077d24... \n", "\n", " created_by_ref created \\\n", "0 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-10-01T01:57:31.229Z \n", "1 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-29T15:10:19.236Z \n", "2 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-28T17:41:12.950Z \n", "3 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-24T21:41:34.797Z \n", "\n", " modified name \\\n", "0 2021-10-15T18:47:18.824Z TeamTNT \n", "1 2021-10-15T15:16:47.329Z Andariel \n", "2 2021-10-25T14:28:10.337Z Ferocious Kitten \n", "3 2021-10-16T02:06:06.404Z IndigoZebra \n", "\n", " description \\\n", "0 [TeamTNT](https://attack.mitre.org/groups/G013... \n", "1 [Andariel](https://attack.mitre.org/groups/G01... \n", "2 [Ferocious Kitten](https://attack.mitre.org/gr... \n", "3 [IndigoZebra](https://attack.mitre.org/groups/... \n", "\n", " aliases \\\n", "0 [TeamTNT] \n", "1 [Andariel, Silent Chollima] \n", "2 [Ferocious Kitten] \n", "3 [IndigoZebra] \n", "\n", " external_references \\\n", "0 [{'source_name': 'mitre-attack', 'url': 'https... \n", "1 [{'source_name': 'mitre-attack', 'url': 'https... \n", "2 [{'source_name': 'mitre-attack', 'url': 'https... \n", "3 [{'source_name': 'mitre-attack', 'url': 'https... \n", "\n", " object_marking_refs \\\n", "0 [marking-definition--fa42a846-8d90-4e51-bc29-7... \n", "1 [marking-definition--fa42a846-8d90-4e51-bc29-7... \n", "2 [marking-definition--fa42a846-8d90-4e51-bc29-7... \n", "3 [marking-definition--fa42a846-8d90-4e51-bc29-7... \n", "\n", " x_mitre_contributors x_mitre_version \n", "0 [Will Thomas, Cyjax] 1.0 \n", "1 [Kyoung-ju Kwak (S2W)] 1.0 \n", "2 [Pooja Natarajan, NEC Corporation India, Manik... 1.0 \n", "3 [Pooja Natarajan, NEC Corporation India, Yoshi... 1.0 " ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "groups_list = []\n", "for t in groups:\n", " groups_list.append(json.loads(t.serialize()))\n", "df = pandas.json_normalize(groups_list)\n", "df[0:4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get All Software\n", "We can extract all Enterprise, Mobile and ICS (Software Malware & Tools)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "software = lift.get_software()" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of Software in ATT&CK\n", "641\n" ] } ], "source": [ "print(\"Number of Software in ATT&CK\")\n", "len(software)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
typeidcreated_by_refcreatedmodifiednamedescriptionlabelsexternal_referencesobject_marking_refsx_mitre_aliasesx_mitre_contributorsx_mitre_platformsx_mitre_versionx_mitre_old_attack_id
0tooltool--f91162cc-1686-4ff8-8115-bf3f61a4cc7aidentity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-09-14T21:45:30.280Z2021-09-21T18:03:13.205ZWevtutil[Wevtutil](https://attack.mitre.org/software/S...[tool][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[Wevtutil][Viren Chaudhari, Qualys, Harshal Tupsamudre, ...[Windows]1.0NaN
1tooltool--11f8d7eb-1927-4806-9267-3a11d4d4d6beidentity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-07-30T15:43:17.770Z2021-10-15T15:49:25.284ZSliver[Sliver](https://attack.mitre.org/software/S06...[tool][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[Sliver][Achute Sharma, Keysight, Ayan Saha, Keysight][Windows, Linux, macOS]1.0NaN
2tooltool--80c815bb-b24a-4b9c-9d73-ff4c075a278didentity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-03-19T13:11:50.666Z2021-04-26T22:35:19.315ZOut1[Out1](https://attack.mitre.org/software/S0594...[tool][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[Out1]NaN[Windows]1.0NaN
3tooltool--03c6e0ea-96d3-4b23-9afb-05055663cf4bidentity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-03-18T14:57:34.628Z2021-04-25T23:30:38.375ZRemoteUtilities[RemoteUtilities](https://attack.mitre.org/sof...[tool][{'source_name': 'mitre-attack', 'url': 'https...[marking-definition--fa42a846-8d90-4e51-bc29-7...[RemoteUtilities]NaN[Windows]1.0NaN
\n", "
" ], "text/plain": [ " type id \\\n", "0 tool tool--f91162cc-1686-4ff8-8115-bf3f61a4cc7a \n", "1 tool tool--11f8d7eb-1927-4806-9267-3a11d4d4d6be \n", "2 tool tool--80c815bb-b24a-4b9c-9d73-ff4c075a278d \n", "3 tool tool--03c6e0ea-96d3-4b23-9afb-05055663cf4b \n", "\n", " created_by_ref created \\\n", "0 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-09-14T21:45:30.280Z \n", "1 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-07-30T15:43:17.770Z \n", "2 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-03-19T13:11:50.666Z \n", "3 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-03-18T14:57:34.628Z \n", "\n", " modified name \\\n", "0 2021-09-21T18:03:13.205Z Wevtutil \n", "1 2021-10-15T15:49:25.284Z Sliver \n", "2 2021-04-26T22:35:19.315Z Out1 \n", "3 2021-04-25T23:30:38.375Z RemoteUtilities \n", "\n", " description labels \\\n", "0 [Wevtutil](https://attack.mitre.org/software/S... [tool] \n", "1 [Sliver](https://attack.mitre.org/software/S06... [tool] \n", "2 [Out1](https://attack.mitre.org/software/S0594... [tool] \n", "3 [RemoteUtilities](https://attack.mitre.org/sof... [tool] \n", "\n", " external_references \\\n", "0 [{'source_name': 'mitre-attack', 'url': 'https... \n", "1 [{'source_name': 'mitre-attack', 'url': 'https... \n", "2 [{'source_name': 'mitre-attack', 'url': 'https... \n", "3 [{'source_name': 'mitre-attack', 'url': 'https... \n", "\n", " object_marking_refs x_mitre_aliases \\\n", "0 [marking-definition--fa42a846-8d90-4e51-bc29-7... [Wevtutil] \n", "1 [marking-definition--fa42a846-8d90-4e51-bc29-7... [Sliver] \n", "2 [marking-definition--fa42a846-8d90-4e51-bc29-7... [Out1] \n", "3 [marking-definition--fa42a846-8d90-4e51-bc29-7... [RemoteUtilities] \n", "\n", " x_mitre_contributors x_mitre_platforms \\\n", "0 [Viren Chaudhari, Qualys, Harshal Tupsamudre, ... [Windows] \n", "1 [Achute Sharma, Keysight, Ayan Saha, Keysight] [Windows, Linux, macOS] \n", "2 NaN [Windows] \n", "3 NaN [Windows] \n", "\n", " x_mitre_version x_mitre_old_attack_id \n", "0 1.0 NaN \n", "1 1.0 NaN \n", "2 1.0 NaN \n", "3 1.0 NaN " ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "software_list = []\n", "for t in software:\n", " software_list.append(json.loads(t.serialize()))\n", "df = pandas.json_normalize(software_list)\n", "df[0:4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get All Relationships\n", "We can also get all relationships from all ATT&CK matrices with one API request." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "relationships = lift.get_relationships()" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of Relationships in ATT&CK\n", "15752\n" ] } ], "source": [ "print(\"Number of Relationships in ATT&CK\")\n", "len(relationships)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
typeidcreated_by_refcreatedmodifiedrelationship_typesource_reftarget_refobject_marking_refsdescriptionexternal_references
0relationshiprelationship--9567076b-2a77-43e4-befd-19556def...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-11-10T09:30:48.753Z2021-11-10T09:30:48.753Zdetectsx-mitre-data-component--3d20385b-24ef-40e1-9f5...attack-pattern--910906dd-8c0a-475a-9cc1-5e029e...[marking-definition--fa42a846-8d90-4e51-bc29-7...NaNNaN
1relationshiprelationship--79fa693d-38b2-4730-8602-1f72eef5...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-11-10T09:30:48.753Z2021-11-10T09:30:48.753Zdetectsx-mitre-data-component--9ce98c86-8d30-4043-ba5...attack-pattern--60d0c01d-e2bf-49dd-a453-f8a9c9...[marking-definition--fa42a846-8d90-4e51-bc29-7...NaNNaN
2relationshiprelationship--ed1c4fff-998f-499d-8a00-cfdee554...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-11-10T09:30:48.753Z2021-11-10T09:30:48.753Zdetectsx-mitre-data-component--9bde2f9d-a695-4344-bfa...attack-pattern--2959d63f-73fd-46a1-abd2-109d7d...[marking-definition--fa42a846-8d90-4e51-bc29-7...NaNNaN
3relationshiprelationship--41c0352d-b377-4fe9-8c3a-67b78a9a...identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b52021-11-10T09:30:48.753Z2021-11-10T09:30:48.753Zdetectsx-mitre-data-component--c0a4a086-cc20-4e1e-b7c...attack-pattern--6836813e-8ec8-4375-b459-abb388...[marking-definition--fa42a846-8d90-4e51-bc29-7...NaNNaN
\n", "
" ], "text/plain": [ " type id \\\n", "0 relationship relationship--9567076b-2a77-43e4-befd-19556def... \n", "1 relationship relationship--79fa693d-38b2-4730-8602-1f72eef5... \n", "2 relationship relationship--ed1c4fff-998f-499d-8a00-cfdee554... \n", "3 relationship relationship--41c0352d-b377-4fe9-8c3a-67b78a9a... \n", "\n", " created_by_ref created \\\n", "0 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z \n", "1 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z \n", "2 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z \n", "3 identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5 2021-11-10T09:30:48.753Z \n", "\n", " modified relationship_type \\\n", "0 2021-11-10T09:30:48.753Z detects \n", "1 2021-11-10T09:30:48.753Z detects \n", "2 2021-11-10T09:30:48.753Z detects \n", "3 2021-11-10T09:30:48.753Z detects \n", "\n", " source_ref \\\n", "0 x-mitre-data-component--3d20385b-24ef-40e1-9f5... \n", "1 x-mitre-data-component--9ce98c86-8d30-4043-ba5... \n", "2 x-mitre-data-component--9bde2f9d-a695-4344-bfa... \n", "3 x-mitre-data-component--c0a4a086-cc20-4e1e-b7c... \n", "\n", " target_ref \\\n", "0 attack-pattern--910906dd-8c0a-475a-9cc1-5e029e... \n", "1 attack-pattern--60d0c01d-e2bf-49dd-a453-f8a9c9... \n", "2 attack-pattern--2959d63f-73fd-46a1-abd2-109d7d... \n", "3 attack-pattern--6836813e-8ec8-4375-b459-abb388... \n", "\n", " object_marking_refs description \\\n", "0 [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN \n", "1 [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN \n", "2 [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN \n", "3 [marking-definition--fa42a846-8d90-4e51-bc29-7... NaN \n", "\n", " external_references \n", "0 NaN \n", "1 NaN \n", "2 NaN \n", "3 NaN " ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "relations_list = []\n", "for t in relationships:\n", " relations_list.append(json.loads(t.serialize()))\n", "df = pandas.json_normalize(relations_list)\n", "df[0:4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get All Data Sources\n", "Now that `data sources` are part of the ATT&CK data model as objects, we can retrieve all that information at once." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "data_sources = lift.get_data_sources()" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of Data Sources in ATT&CK\n", "38\n" ] } ], "source": [ "print(\"Number of Data Sources in ATT&CK\")\n", "len(data_sources)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],\n", " 'modified': '2021-10-20T15:05:19.275Z',\n", " 'name': 'Internet Scan',\n", " 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',\n", " 'type': 'x-mitre-data-source',\n", " 'id': 'x-mitre-data-source--38fe306c-bdec-4f3d-8521-b72dd32dbd17',\n", " 'description': 'Information obtained (commonly via active network traffic probes or web crawling) regarding various types of resources and servers connected to the public Internet',\n", " 'created': '2021-10-20T15:05:19.275Z',\n", " 'external_references': [{'url': 'https://attack.mitre.org/datasources/DS0035',\n", " 'external_id': 'DS0035',\n", " 'source_name': 'mitre-attack'}],\n", " 'x_mitre_version': '1.0',\n", " 'x_mitre_platforms': ['PRE'],\n", " 'x_mitre_collection_layers': ['OSINT'],\n", " 'x_mitre_contributors': []}" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_sources[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get All Data Components\n", "Now that `data components` are also part of the ATT&CK data model as objects, we can retrieve all that information at once." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "data_components = lift.get_data_components()" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of data components in ATT&CK\n", "109\n" ] } ], "source": [ "print(\"Number of data components in ATT&CK\")\n", "len(data_components)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],\n", " 'modified': '2021-10-20T15:05:19.275Z',\n", " 'id': 'x-mitre-data-component--cc150ad8-ecfa-4340-9aaa-d21165873bd4',\n", " 'description': 'Logged domain name system (DNS) data highlighting timelines of domain to IP address resolutions (ex: passive DNS)',\n", " 'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',\n", " 'name': 'Passive DNS',\n", " 'created': '2021-10-20T15:05:19.275Z',\n", " 'type': 'x-mitre-data-component',\n", " 'x_mitre_version': '1.0',\n", " 'x_mitre_data_source_ref': 'x-mitre-data-source--dd75f457-8dc0-4a24-9ae5-4b61c33af866'}" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_components[0]" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.5" } }, "nbformat": 4, "nbformat_minor": 4 }