For a new project I needed a tool that could scan a web server for the http status code of different URLs and have the results listed in a easily parseable result. The URLs are typical Linux resources (f.e. the password file, the hostname, services file, …) that could lead to disclosing sensitive system information.

There are already a number of tools that can achieve this but none really provided the output that I needed. Hence I wrote my own small script.

You can find it on Github : https://github.com/cudeso/tools/tree/master/twus

It’s a python script that needs one necessary parameter, the URL and two optional parameters. The URL can contain a port number if necessary. So both http://www.myhost.com/ and http://www.myhost.com:995/ are valid.

usage: twus.py [-h] [-r RESOURCES] [-v] url

Tiny Web Url Scanner

positional arguments:
  url                   The URL to scan (include http://)

optional arguments:
  -h, --help            show this help message and exit
  -r RESOURCES, --resources RESOURCES
                        File containing the resources to check (default:
                        twus.input)
  -v, --verbose         Add verbose output (default: False)

Start scanning
 Base URL: http://www.changethis.com/ 

"Code", "URL", "Server", "Last Modified", "Content Type", "Cache Control"
"404", "http://www.changethis.com/tmp", "", "", "", ""
"200", "http://www.changethis.com/home", "Apache/2 (Linux)", "", "text/html; charset=UTF-8", ""
"404", "http://www.changethis.com/etc/hostname", "", "", "", ""
"404", "http://www.changethis.com/etc/motd", "", "", "", ""
"404", "http://www.changethis.com/etc/issue", "", "", "", ""
"404", "http://www.changethis.com/var/run/utmp", "", "", "", ""

Scan finished

The file containing the resources lists one resource per line. You don’t need to pay attention if it starts with / or not. If the resource starts with a / then it’s stripped. If the URL is not ended with a /, then there is a / added.

twus.input, the default resources file, contains mostly Linux interesting resources.

/tmp
/home
/etc/hostname
/etc/motd
/etc/issue
/etc/passwd
/etc/syslog.conf
/etc/mailname
/etc/mail.rc