Doing open source intel with recon-ng – part 1

Recon-ng

What is recon-ng?

recon-ng is a tool for open source reconnaissance. Reconnaissance is the first phase in a penetration test and it is the act of gathering preliminary data or intelligence on your target.

Recon-ng has a look and feel similar to the Metasploit Framework and provides an easy to use interface to gather open source intelligence.

This is a post on doing open source intel with recon-ng. The post is split in two parts :


Recon-ng

Installation

The installation of recon-ng is very easy on Ubuntu Linux.

git clone https://LaNMaSteR53@bitbucket.org/LaNMaSteR53/recon-ng.git
cd recon-ng
sudo pip install -r REQUIREMENTS

This will install the latest version of recon-ng. You can then start it with

./recon-ng

How do you use recon-ng?

Open source intel with recon-ng

The best way to demonstrate recon-ng is via a use-case. In this example I will gather as much open source information as possible starting with my company domain name (c[u]de[s]o.be).

Recon-ng is highly database-driven. This means that all the operations are done starting with the information that is already available in the database.

But if you start with an empty database you need to inject a keyword somewhere to get recon-ng started …

Start with a workspace and one domain

I first start with a new workspace. This is not entirely necessary but it keeps the results cleanly contained in one single container. With the use of workspaces you can run multiple different recon operations without having the results getting mixed up with each other.

[recon-ng][default] > workspaces add c[u]de[s]o
[recon-ng]de[s]o] >

My starting point is a domain so I have to add this information manually to the database.

[recon-ng]de[s]o] > add domains c[u]de[s]o.be

[recon-ng]de[s]o] > show domains

  +----------------------------------+
  | rowid |   domain  |    module    |
  +----------------------------------+
  | 1     | c[u]de[s]o.be | user_defined |
  +----------------------------------+

[*] 1 rows returned

From domains to contacts

I now want to search for information starting with only domain information. Recon-ng has an easy way to get all the modules that can work further on domain information.

[recon-ng]de[s]o] > search domains-
[*] Searching for 'domains-'...

  Recon
  -----
    recon/domains-contacts/metacrawler
    recon/domains-contacts/pgp_search
    recon/domains-contacts/salesmaple
    recon/domains-contacts/whois_pocs
    recon/domains-credentials/pwnedlist/account_creds

By using SEARCH domains- I get everything that adds information starting from a domain. You can also use the search feature the other way around with SEARCH -domains. This will list every module that results in domain information.

I now use the PGP search module to get contact information.

[recon-ng]de[s]o] > use recon/domains-contacts/pgp_search
[recon-ng]de[s]o][pgp_search] >

Similar to the Metasploit Framework you can get the list of options with the SET command

[recon-ng]de[s]o][pgp_search] > set
Sets module options

Usage: set <option> <value>

  Name    Current Value  Required  Description
  ------  -------------  --------  -----------
  SOURCE  default        yes       source of input (see 'show info' for details)

Source option

This module has only one option, the SOURCE option. This option is something that you’ll also see in the other modules.

Remember that earlier I mentioned that recon-ng is highly database-driven. This option allows you to influence this database-driven behavior. Instead of using the database information as a starting point you can provide your own information as a starting point.

In an earlier step I added a domain manually. I could have skipped that step and used the PGP module directly, feeding it the domain manually as a source. By doing it that way however I would have lost the conceptual relationship “domain -> contacts”.

It is easy to check what information a module needs for a starting point with SHOW INFO

[recon-ng]de[s]o][pgp_search] > show info

      Name: PGP Key Owner Lookup
      Path: modules/recon/domains-contacts/pgp_search.py
    Author: Robert Frost (@frosty_1313, frosty[at]unluckyfrosty.net)

Description:
  Searches the MIT public PGP key server for email addresses of the given domain. Updates the
  'contacts' table with the results.

Options:
  Name    Current Value  Required  Description
  ------  -------------  --------  -----------
  SOURCE  default        yes       source of input (see 'show info' for details)

Source Options:
  default        SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL
  <string>       string representing a single input
  <path>         path to a file containing a list of inputs
  query <sql>    database query returning one column of inputs

Comments:
  * Inspiration from theHarvester.py by Christan Martorella: cmarorella[at]edge-seecurity.com

The information that is needed to start the crawling of the module is set in the default query string. In this case it is “SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL” meaning that it needs a domain to start with. You can change the default (database) behavior to using a string as input or also a file. The latter is very useful if you have multiple starting sources for a module (in this case for example imagine a case where you need to conduct a recon for a customer provided domain list).

Run the module and view the results

Running the module is easy with RUN

[recon-ng]de[s]o][pgp_search] > run

---------
c[u]de[s]o.BE
---------
[*] Koen Van Impe (koen.van[i]m[p]e@c[u]de[s]o.be)
[*] c[u]de[s]o sales (sales@c[u]de[s]o.be)

-------
SUMMARY
-------
[*] 2 total (2 new) contacts found.

The results will immediately be added to the database. You can have a look at the current content of the database with SHOW DASHBOARD

[recon-ng]de[s]o][pgp_search] > show dashboard

  +------------------------------------------+
  |             Activity Summary             |
  +------------------------------------------+
  |               Module              | Runs |
  +------------------------------------------+
  | recon/domains-contacts/pgp_search | 1    |
  +------------------------------------------+


  +----------------------------+
  |      Results Summary       |
  +----------------------------+
  |     Category    | Quantity |
  +----------------------------+
  | Domains         | 1        |
  | Companies       | 0        |
  | Netblocks       | 0        |
  | Locations       | 0        |
  | Vulnerabilities | 0        |
  | Ports           | 0        |
  | Hosts           | 0        |
  | Contacts        | 2        |
  | Credentials     | 0        |
  | Leaks           | 0        |
  | Pushpins        | 0        |
  | Profiles        | 0        |
  | Repositories    | 0        |
  +----------------------------+

This shows I have information on one domain and two contacts. What if I have forgotten the contact details? You can display them (along with information from the other categories) via SHOW CONTACTS

[recon-ng]de[s]o][pgp_search] > show contacts

  +-----------------------------------------------------------------------------------------------------------------------------+
  | rowid | first_name | middle_name | last_name |         email          |        title        | region | country |   module   |
  +-----------------------------------------------------------------------------------------------------------------------------+
  | 1     | Koen       | Van         | Impe      | koen.van[i]m[p]e@c[u]de[s]o.be | PGP key association |        |         | pgp_search |
  | 2     | c[u]de[s]o     |             | sales     | sales@c[u]de[s]o.be        | PGP key association |        |         | pgp_search |
  +-----------------------------------------------------------------------------------------------------------------------------+

[*] 2 rows returned

Expanding contact information

I’d now like to expand the contact information. What is available with SEARCH CONTACTS-?

[recon-ng]de[s]o][hibp_paste] > search contacts-
[*] Searching for 'contacts-'...

  Recon
  -----
    recon/contacts-contacts/mailtester
    recon/contacts-contacts/mangle
    recon/contacts-contacts/unmangle
    recon/contacts-credentials/hibp_breach
    recon/contacts-credentials/hibp_paste
    recon/contacts-credentials/pwnedlist
    recon/contacts-domains/migrate_contacts
    recon/contacts-profiles/fullcontact
 

The hibp_ modules can get me useful credential information from (previous) account breaches that is made available via https://haveibeenpwned.com/. First I want to have an overview of what breaches contain useful pointers for my search

[recon-ng]de[s]o][hibp_breach] > use     recon/contacts-credentials/hibp_breach
[recon-ng]de[s]o][hibp_breach] > run
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Breach found! Seen in the Adobe breach that occurred on 2013-10-04.
[*] sales@c[u]de[s]o.be => Not Found.

-------
SUMMARY
-------
[*] 1 total (1 new) credentials found.
[*] 1 total (0 new) contacts found.

This search returns one breach (the Adobe hack) that has useful account information.

I now search for the pasties where this information was stored. These pasties sometimes can hold extra useful information. By default hibp_paste will attempt to download the pastie.

[recon-ng]de[s]o][hibp_paste] > set
Sets module options

Usage: set <option> <value>

  Name      Current Value  Required  Description
  --------  -------------  --------  -----------
  DOWNLOAD  True           yes       download pastes
  SOURCE    default        yes       source of input (see 'show info' for details)

You can disable this with the option SET DOWNLOAD False.

[recon-ng]de[s]o][hibp_breach] > use recon/contacts-credentials/hibp_paste
[recon-ng]de[s]o][hibp_paste] > run
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-12-08T05:12:00Z (http://pastebin.com/raw.php?i=C4b1t5Db).
[*] Paste could not be downloaded (http://pastebin.com/raw.php?i=C4b1t5Db).
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-12-08T04:12:00Z (http://pastebin.com/raw.php?i=h6b4BRmt).
[*] Paste could not be downloaded (http://pastebin.com/raw.php?i=h6b4BRmt).
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-26T09:11:00Z (http://pastebin.com/raw.php?i=zc3vCANP).
[*] Paste could not be downloaded (http://pastebin.com/raw.php?i=zc3vCANP).
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T22:11:00Z (http://pastebin.com/raw.php?i=0tbNjD4h).
[*] Paste could not be downloaded (http://pastebin.com/raw.php?i=0tbNjD4h).
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T13:11:00Z (http://pastebin.com/raw.php?i=Bi7mv9Kw).
[*] Paste could not be downloaded (http://pastebin.com/raw.php?i=Bi7mv9Kw).
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T11:11:00Z (http://pastebin.com/raw.php?i=j0kXGE6w).
[*] Paste could not be downloaded (http://pastebin.com/raw.php?i=j0kXGE6w).
[*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T01:11:00Z (http://pastebin.com/raw.php?i=c5a4bb1z).
[*] Paste could not be downloaded (http://pastebin.com/raw.php?i=c5a4bb1z).
[*] sales@c[u]de[s]o.be => Not Found.

-------
SUMMARY
-------
[*] 1 total (0 new) credentials found.
[*] 1 total (0 new) contacts found.

The module has found a couple of pasties but unfortunately they have already been deleted.

Extending the credentials

What options are available to extend the credentials? I search for the modules that can use credentials as input and output with SEARCH CREDENTIALS:

[recon-ng]de[s]o] > search credentials
[*] Searching for 'credentials'...

  Recon
  -----
    recon/contacts-credentials/hibp_breach
    recon/contacts-credentials/hibp_paste
    recon/contacts-credentials/pwnedlist
    recon/credentials-credentials/adobe
    recon/credentials-credentials/bozocrack
    recon/credentials-credentials/hashes_org
    recon/credentials-credentials/leakdb
    ...

For the purpose of this demo I assume a hash was found in a previous run. For this example I add a hash (that could for example have been found via the pasties-information) manually to a record. In order to do this I use the QUERY command

[recon-ng]de[s]o] > query update credentials set hash = '739c5b1cd5681e668f689aa66bcc254c'

[recon-ng]de[s]o] > show credentials

  +----------------------------------------------------------------------------------------------------------+
  | rowid |        username        | password |               hash               | type | leak |    module   |
  +----------------------------------------------------------------------------------------------------------+
  | 1     | koen.van[i]m[p]e@c[u]de[s]o.be |          | 739c5b1cd5681e668f689aa66bcc254c |      |      | hibp_breach |
  +----------------------------------------------------------------------------------------------------------+

[*] 1 rows returned

I’ll query https://hashes.org for matches on this hash. In order to do so I have to supply an API key.

Adding API keys for modules

Some modules will access public resources via an API and they require an API key. You have to add this API key to recon-ng with the command keys add.

[recon-ng]de[s]o] > use recon/credentials-credentials/hashes_org

[recon-ng]de[s]o][hashes_org] > keys add hashes_api replace_with_my_key
[*] Key 'hashes_api' added.

Now that an API key has been set we can use the module to extend the credential information.

[recon-ng]de[s]o][hashes_org] > run
[*] 739c5b1cd5681e668f689aa66bcc254c (MD5X5PLAIN) => test
[recon-ng]de[s]o][hashes_org] > show credentials

  +----------------------------------------------------------------------------------------------------------------+
  | rowid |        username        | password |               hash               |    type    | leak |    module   |
  +----------------------------------------------------------------------------------------------------------------+
  | 1     | koen.van[i]m[p]e@c[u]de[s]o.be | test     | 739c5b1cd5681e668f689aa66bcc254c | MD5X5PLAIN |      | hibp_breach |
  +----------------------------------------------------------------------------------------------------------------+

[*] 1 rows returned

The above command shows that a match with the hash has been found in the database of hashes.org. The matching password is automatically added to the credentials table.

Social media profiles

With the help of the module for Fullcontact I’m able to get an overview of other available social media profiles for the accounts that were previously found. Note that this module also requires an API key.

[recon-ng]de[s]o][linkedin] > use recon/contacts-profiles/fullcontact
[recon-ng]de[s]o][fullcontact] > run
[!] FrameworkException: API key 'fullcontact_api' not found. Add API keys with the 'keys add' command.
[recon-ng]de[s]o][fullcontact] > keys
add     delete  list
[recon-ng]de[s]o][fullcontact] > keys add fullcontact_api myfullcontact_api
[*] Key 'fullcontact_api' added.
[recon-ng]de[s]o][fullcontact] > run
[*] Koen Van Impe - koen.van[i]m[p]e@c[u]de[s]o.be
[*] Brugge
[*] 812988488 - Facebook (https://www.facebook.com/812988488)
[*] 1100812 - Foursquare (https://foursquare.com/user/1100812)
[*] c[u]de[s]o - Flickr (https://www.flickr.com/people/c[u]de[s]o)
[*] c[u]de[s]o - Gravatar (https://gravatar.com/c[u]de[s]o)
[*] c[u]de[s]o - Twitter (https://twitter.com/c[u]de[s]o)
[*] Confidence: 89%
[*] sales@c[u]de[s]o.be - Searched within last 24 hours. No results found for this Id.

-------
SUMMARY
-------
[*] 5 total (5 new) profiles found.
[*] 1 total (1 new) contacts found.

You can extend the profiling information for the different user accounts with other modules. Just do a search for everything that extends profiles with SEARCH profiles.

 
[recon-ng]de[s]o][profiler] > search profiles
[*] Searching for 'profiles'...

  Recon
  -----
    recon/companies-profiles/bing_linkedin
    recon/contacts-profiles/fullcontact
    recon/profiles-contacts/dev_diver
    recon/profiles-contacts/linkedin
    recon/profiles-profiles/linkedin_crawl
    recon/profiles-profiles/namechk
    recon/profiles-profiles/profiler
    recon/profiles-profiles/twitter
    recon/profiles-repositories/github_repos

End of part 1

This is the first part of a post on doing open source intel with recon-ng. This post focused on gathering open source information for user accounts. The second part on recon-ng focuses on gathering domain and host information.

Leave a Reply

Your email address will not be published. Required fields are marked *