Intelligence Gathering

General

This section defines the Intelligence Gathering activities of a penetration test. The purpose of this document is to provide a (living?) document designed specifically for the pentester performing reconnaissance against a target (typically corporate, military, or related). The document details the thought process and goals of pentesting reconnaissance, and when used properly, helps the reader to produce a highly strategic plan for attacking a target.

Intelligence Gathering

What is it

• Intelligence Gathering is performing reconnaissance against a target to gather as much information as possible to be utilized when penetrating the target during the vulnerability assessment and exploitation phases. The more information you are able to gather during this phase, the more vectors of attack you may be able to use in the future.
• Open source intelligence (OSINT) is a form of intelligence collection management that involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence. [1]

Why do it

• We perform Open Source Intelligence gathering to determine various entry points into an organization. These entry points can be physical, electronic, and/or human. Many companies fail to take into account what information about themselves they place in public and how this information can be used by a determined attacker. On top of that many employees fail to take into account what information they place about themselves in public and how that information can be used to to attack them or their employer.

What is it not

• [Needs Content]

Target Selection

Consider any Rules of Engagement limitations

• Rules of Engagement
  • Typical Reconnaissance rules
  • Defining your own rules

OSINT

Open Source Intelligence (OSINT) takes three forms; Passive, Semi-passive, and Active.

• Passive Information Gathering: Passive Information Gathering is generally only useful if there is a very clear requirement that the information gathering activities never be detected by the target. This type of profiling is technically difficult to perform as we are never sending any traffic to the target organization neither from one of our hosts or “anonymous” hosts or services across the Internet. This means we can only use and gather archived or stored information. As such this information can be out of date or incorrect as we are limited to results gathered from a third party.

• Semi-passive Information Gathering: The goal for semi-passive information gathering is to profile the target with methods that would appear like normal Internet traffic and behavior. We query only the published name servers for information, we aren’t performing in-depth reverse lookups or brute force DNS requests, we aren’t searching for “unpublished” servers or directories. We aren’t running network level portscans or crawlers and we are only looking at metadata in published documents and files; not actively seeking hidden content. The key here is not to draw attention to our activities. Post mortem the target may be able to go back and discover the reconnaissance activities but they shouldn’t be able to attribute the activity back to anyone.

• Active Information Gathering: Active information gathering should be detected by the target and suspicious or malicious behavior. During this stage we are actively mapping network infrastructure (think full port scans nmap –p1-65535), actively enumerating and/or vulnerability scanning the open services, we are actively searching for unpublished directories, files, and servers. Most of this activity falls into your typically “reconnaissance” or “scanning” activities for your standard pentest.

Corporate

Physical

Locations

Per location listing of full address, ownership, associated records (city, tax, legal, etc), Full listing of all physical security measures for the location (camera placements, sensors, fences, guard posts, entry control, gates, type of identification, supplier’s entrance, physical locations based on IP blocks/geolocation services, etc…

• Owner
• Land/tax records
• Shared/individual
• Timezones
• Hosts / NOC

Pervasiveness

• [Need Content]

Relationships

Business partners, customs, suppliers, analysis via whats openly shared on corporate web pages, rental companies, etc

• Relationships
• Shared office space
• Shared infrastructure
• Rented / Leased Equipment

Logical

Accumulated information for partners, clients and competitors: For each one, a full listing of the business name, business address, type of relationship, basic financial information, basic hosts/network information.

Business Partners

• Target’s advertised business partners. Sometimes advertised on main www.

Business Clients

• Target’s advertised business clients. Sometimes advertised on main www.

Competitors

• Who are the target’s competitors. This may be simple, Ford vs Chevy, or may require much more analysis.

Touchgraph

• [Need Content]

Hoovers profile

• What: a semi-open source intelligence resource (paid subscriptions usually). Such sources specialize in gathering business related information on companies, and providing a “normalized” view on the business.
• Why: The information includes physical locations, competitive landscape, key personnel, financial information, and other business related data (depending on the source). This can be used to create a more accurate profile of the target, and identify additional personnel and 3rd parties which can be used in the test.
• How: Simple search on the site with the business name provide the entire profile of the company and all the information that is available on it. Its recommended to use a couple of sources in order to cross reference them and make sure you get the most up-to-date information. (paid for service).

Product line

• [Need Content]

Market Vertical

• Which industry the target resides in. i.e. financial, defense, agriculture, government, etc

Marketing accounts

• [Need Content]

Meetings

• Meeting Minutes published?
• Meetings open to public?

Significant company dates

• Board meetings
• Holidays
• Anniversaries
• Product/service launch

Job openings

• By viewing a list of job openings at an organization (usually found in a ‘careers’ section of their website), you can determine types of technologies used within the organization. One example would be if an organization has a job opening for a Senior Solaris Sysadmin then it is pretty obvious that the organization is using Solaris systems. Other positions may not be as obvious by the job title, but an open Junior Network Administrator position may say something to the effect of ‘CCNA preferred’ or ‘JNCIA preferred’ which tells you that they are either using Cisco or Juniper technologies.

Charity affiliations

• [Need Content]

Court records

• [Need Content]

Political donations

• [Need Content]

Professional licenses or registries

• [Need Content]

Org Chart

Position identification

• Important people in the organization
• Individuals to specifically target

Transactions

• [Need Content]

Affiliates

• [Need Content]

Electronic

Document Metadata

• What it is? Metadata or meta-content provides information about the data/document in scope. It can have information such as author/creator name, time and date, standards used/referred, location in a computer network (printer/folder/directory path/etc. info), geo-tag etc. For an image its’ metadata can contain color, depth, resolution, camera make/type and even the co-ordinates and location information.
• Why you would do it? Metadata is important because it contains information about the internal network, user-names, email addresses, printer locations etc. and will help to create a blueprint of the location. It also contains information about software used in creating the respective documents. This can enable an attacker to create a profile and/or perform targeted attacks with internal knowledge on the networks and users.
• How you would do it? There are tools available to extract the metadata from the file (pdf/word/image) like FOCA (GUI-based), metagoofil (python-based), meta-extractor, exiftool (perl-based). These tools are capable of extracting and displaying the results in different formats as HTML, XML, GUI, JSON etc. The input to these tools is mostly a document downloaded from the public presence of the ‘client’ and then analyzed to know more about it. Whereas FOCA helps you search documents, download and analyzes all through its GUI interface.

Marketing Communications

• [Need Content]

Infrastructure Assets

[!!This section needs to be added to mindmap!!]

Network blocks owned

• [Need Content]

Email addresses

• [Need Content]

External infrastructure profile

• [Need Content]

Technologies used

• [Need Content]

Purchase agreements

• [Need Content]

Remote access

• [Need Content]

Defense technologies

• [Need Content]

Human capability

• [Need Content]

Financial

Reporting

• [Need Content]

Market analysis

• [Need Content]

Trade capital

• [Need Content]

Value history

• [Need Content]

EDGAR (SEC)

• What is it: EDGAR (the Electronic Data Gathering, Analysis, and Retrieval system) is a database of the U.S. Security and Exchanges Commission (SEC) that contains registration statements, periodic reports, and other information of all companies (both foreign and domestic) who are required by law to file.
• Why do it: EDGAR data is important because, in additional to financial information, it identifies key personnel within a company that may not be otherwise notable from a company’s website or other public presence. It also includes statements of executive compensation, names and addresses of major common stock owners, a summary of legal proceedings against the company, economic risk factors, and other potentially interesting data.
• How to obtain: The information is available on the SEC’s EDGAR website (http://www.sec.gov/edgar.shtml). Reports of particular interest include the 10-K (annual report) and 10-Q (quarterly report).

Individual

Employee

History

• Court Records
  • What is it: Court records are all the public records related to criminal and/or civil complaints, lawsuits, or other legal actions for or against a person or organization of interest.
  • Why you would do it: Court records could potentially reveal sensitive information related to an individual employee or the company as a whole. This information could be useful by itself or may be the driver for gaining additional information. It could also be used for social engineering or other purposes later on in the penetration test.
  • How you would do it: Much of this information is now available on the Internet via publicly available court websites and records databases. Some additional information may be available via pay services such as LEXIS/NEXIS. Some information may be available via records request or in person requests.
• Political Donations
  • What is it: Political donations are an individual’s personal funds directed to specific political candidates, political parties, or special interest organizations.
  • Why you would do it: Information about political donations could potentially reveal useful information related to an individual. This information could be used as a part of social network analysis to help draw connections between individuals and politicians, political candidates, or other political organizations. It could also be used for social engineering or other purposes later on in the penetration test.
  • How you would do it: Much of this information is now available on the Internet via publicly available websites (i.e., http://www.opensecrets.org/) that track political donations by individual. Depending upon the laws of a given state, donations over a certain amount are usually required to be recorded.
• Professional licenses or registries
  • What is it: Professional licenses or registries are repositories of information that contain lists of members and other related information for individuals who have attained a particular license or some measure of specific affiliation within a community.
  • Why you would do it: Information about professional licenses could potentially reveal useful information related to an individual. This information could be used to validate an individual's trustworthiness (do they really have a particular certification as they claim) or as a part of social network analysisto help draw connections between individuals and other organizations. It could also be used for social engineering or other purposes later on in the penetration test.
  • How you would do it: Much of this information is now available on the Internet via publicly available websites. Typically, each organization maintains their own registry of information that may be available online or may require additional steps to gather.

Social Network (SocNet) Profile

• Metadata Leakage
  • Location awareness via Photo Metadata
• Tone
  • Expected deliverable: subjective identification of the tone used in communications – aggressive, passive, appealing, sales, praising, dissing, condescending, arrogance, elitist, underdog, leader, follower, mimicking, etc…
• Frequency
  • Expected deliverable: Identification of the frequency of publications (once an hour/day/week, etc…). Additionally - time of day/week in which communications are prone to happen.
• Location awareness
  • Bing Map Apps
  • Foursquare
  • Google Latitude
  • Yelp
  • Gowalla
• Social Media Presence
  • What sites do they use?

Internet Presence

• Email Address
  • What it is? Email addresses are the public mail box ids of the users.
  • Why you would do it? Email address harvesting or searching is important because it serves multiple purposes - provides a probable user-id format which can later be brute-forced for access but more importantly it helps sending targeted spams and even to automated bots. These spam emails can contain exploits, malware etc. and can be addressed with specific content particularly to a user.
  • How you would do it? Email addresses can be searched and extracted from various websites, groups, blogs, forums, social networking portals etc. These email addresses are also available from various tech support websites. There are harvesting and spider tools to perform search for email addresses mapped to a certain domain (if needed).
• Personal Handles/Nicknames
• Personal Domain Names registered
• Assigned Static IPs/Netblocks

Physical Location

• Physical Location
  • Can you derive the target's physical location

Mobile Footprint

• Phone #
• Device type
• Use
• Installed applications
• Owner/administrator

"For Pay" Information

• Background Checks
• For Pay Linked-In
• LEXIS/NEXIS

Covert Gathering

Corporate

On-Location Gathering

• [Need Content]

Physical security inspections

• [Need Content]

Wireless scanning / RF frequency scanning

• [Need Content]

Employee behavior training inspection

• [Need Content]

Accessible/adjacent facilities (shared spaces)

• [Need Content]

Dumpster diving

• [Need Content]

Types of equipment in use

• [Need Content]

Offsite Gathering

Data center locations

• [Need Content]

Network provisioning/provider

• [Need Content]

HUMINT

Human intelligence complements the more passive gathering on the asset as it provides information that could not have been obtained otherwise, as well as add more “personal” perspectives to the intelligence picture (feelings, history, relationships between key individuals, “atmosphere”, etc...)

The methodology of obtaining human intelligence always involves direct interaction - whether physical, or verbal. Gathering should be done under an assumed identity, that would be created specifically to achieve optimal information exposure and cooperation from the asset in question.

Additionally, intelligence gathering on more sensitive targets can be performed by utilizing observation only - again, either physically on location, or through electronic/remote means (CCTV, webcams, etc...). This is usually done in order to establish behavioral patterns (such as frequency of visitations, dress code, access paths, key locations that may provide additional access such as coffee shops).

Results

Key Employees

• [Need Content]

Partners/Suppliers

• [Need Content]

Social Engineering

• [Need Content]

Footprinting

WHAT IT IS: External information gathering, also known as footprinting, is a phase of information gathering that consists of interaction with the target in order to gain information from a perspective external to the organization.

WHY: Much information can be gathered by interacting with targets. By probing a service or device, you can often create scenarios in which it can be fingerprinted, or even more simply, a banner can be procured which will identify the device. This step is necessary to gather more information about your targets. Your goal, after this section, is a prioritized list of targets.

External Footprinting

Identify Customer External Ranges

One of the major goals of intelligence gathering during a penetration test is to determine hosts which will be in scope. There are a number of techniques which can be used to identify systems, including using reverse DNS lookups, DNS bruting, WHOIS searches on the domains and the ranges. These techniques and others are documented below.

Passive Reconnaissance

WHOIS Lookups

For external footprinting, we first need to determine which one of the WHOIS servers contains the information we're after. Given that we should know the TLD for the target domain, we simply have to locate the Registrar that the target domain is registered with.

WHOIS information is based upon a tree hierarchy. ICANN (IANA) is the authoritative registry for all of the TLDs and is a great starting point for all manual WHOIS queries.

WHOIS lookup

• ICANN - http://www.icann.org
• IANA - http://www.iana.com
• NRO - http://www.nro.net
• AFRINIC - http://www.afrinic.net
• APNIC - http://www.apnic.net
• ARIN - http://ws.arin.net
• LACNIC - http://www.lacnic.net
• RIPE - http://www.ripe.net

Once the appropriate Registrar was queried we can obtain the Registrant information. There are numerous sites that offer WHOIS information; however for accuracy in documentation, you need to use only the appropriate Registrar.

• InterNIC - http://www.internic.net/ http://www.internic.net]

BGP looking glasses

It is possible to identify the Autonomous System Number (ASN) for networks that participate in Border Gateway Protocol (BGP). Since BGP route paths are advertised throughout the world we can find these by using a BGP4 and BGP6 looking glass.

• BGP4 - http://www.bgp4.as/looking-glasses
• BPG6 - http://lg.he.net/

Active Footprinting

Port Scanning

Port scanning techniques will vary based on the amount of time available for the test, and the need to be stealthy. If there is zero knowledge of the systems, a fast ping scan can be used to identify systems. In addition, a quick scan without ping verification (-PN in nmap) should be run to detect the most common ports avialable. Once this is complete, a more comprehensive scan can be run. Some testers check for only open TCP ports, make sure to check UDP as well. The http://nmap.org/nmap_doc.html document details port scan types. Nmap ("Network Mapper") is the de facto standard for network auditing/scanning. Nmap runs on both Linux and Windows.

Nmap has dozens of options available. Since this section is dealing with port scanning, we will focus on the commands required to perform this task. It is important to note that the commands utilized depend mainly on the time and number of hosts being scanned. The more hosts or less time that you have to perform this tasks, the less that we will interrogate the host. This will become evident as we continue to discuss the options.

Based on the IP set being assessed you would want to scan both the TCP and UDP ports across the range 1 to 65535. The command that will be utilized is as follows:

nmap -A -PN -sU -sS -T2 -v -p 1-65535 <client ip range>/<CIDR> or <Mask> -oA NMap_FULL_<client ip range>

nmap -A -PN -sU -sS -T2 -v -p 1-65535 client.com -oA NMap_FULL_client

Also, IPv6 should be tested.

Banner Grabbing

Banner Grabbing is an enumeration technique used to glean information about computer systems on a network and the services running its open ports. Banner grabbing is used to identify network the version of applications and operating system that the target host are running.

Banner grabbing is usually performed on Hyper Text Transfer Protocol (HTTP), File Transfer Protocol (FTP), and Simple Mail Transfer Protocol (SMTP); ports 80, 21, and 25 respectively. Tools commonly used to perform banner grabbing are Telnet, nmap, and Netcat.

HTTP

JUNK / HTTP/1.0
HEAD / HTTP/9.3
OPTIONS / HTTP/1.0
HEAD / HTTP/1.0

SNMP Sweeps

SNMP sweeps are performed too as they offer tons of information about a specific system. The SNMP protocol is a stateless, datagram oriented protocol. Unfortunately SNMP servers don't respond to requests with invalid community strings and the underlying UDP protocol does not reliably report closed UDP ports. This means that "no response" from a probed IP address can mean either of the following:

• machine unreachable
• SNMP server not running
• invalid community string
• the response datagram has not yet arrived

SMTP Bounce Back

SMTP bounce back, also called a Non-Delivery Report/Receipt (NDR), a (failed) Delivery Status Notification (DSN) message, a Non-Delivery Notification (NDN) or simply a bounce, is an automated electronic mail message from a mail system informing the sender of another message about a delivery problem. This can be used to assist an attacker in fingerprint the SMTP server as SMTP server information, including software and versions, may be included in a bounce message.

This can be done by simply creating a bogus address within the target's domain. For instance, asDFADSF_garbage_address@target.com could be used to test target.com. Gmail provides full access to the headers, making it an easy choice for testers.

Identify Protection Mechanisms

User Protections

AV/Spam Filtering Software

• SW Configuration which limit exploitability can be considered antispam / antiAV