The Linux password file, /etc/passwd

The /etc/passwd file stores crucial information which is required during login on Linux systems.

Fields

A line in /etc/passwd is one entry for a user account. The fields are separated by a colon (:).

The format is as follows (note that for the purpose of formatting the display, the line is split. A real /etc/passwd file would have all the data on one line).

newusername:x:1050:1001
     1      2   3    4   
 :NewUser,Roomnr,123,456,user@newdomain.com
        5
 :/home/newusername:/bin/sh
        6               7

 1: username
 2: password 
 3: the user id
 4: the group id
 5: user information (display with 'finger')
 6: the home directory
 7: the shell 

If the password field (2) contains an X then the encrypted password is stored in /etc/shadow.
If it contains an * then the account is disabled.

User IDs

Every user on a system must have an ID, called an UID (field 3). The super user, or root user, is assigned UID 0. UIDs of 1 to 99 are reserved for predefined accounts. The IDs from 100 to 999 are reserved for system accounts.

If you spot a user account with UID of 0 and it’s not root then this might be the sign of a break-in.
An easy way to look for UID 0 accounts is with

egrep ':0+:' /etc/passwd

Shadow file

Typically the passwd file is readable by world. This would mean that if passwords would also be stored in the passwd file these passwords are accessible by every account. To avoid this, passwords are stored in another file /etc/shadow that is only accessible by the root user.

Similarly to the passwd file, this file contains one entry per account and all fields are separated by a colon (:). The format is as follows

newusername:$verylongstrong$:15086:0:99999:7:::
   1              2             3  4   5   6

 1: username
 2: encrypted password 
 3: number of days since last change
 4: minimum of days required between password changes
 5: the maximum of days a password is valid
 6: the number of days, before the expiration of a password, that a user is warned  

If the encrypted password field starts with $ then this means it was generated with something else than DES. For example $1$ indicates it is generated with MD5, $6$ is SHA-512.

Edit the password or shadow file

Do not edit the passwd file or shadow file directly with vi.
You should use the special purpose tools as vipw or vigr to change these files.

Nmap scan through TOR

TOR (https://www.torproject.org/) is a great project if you want to take care of your privacy. You can use TOR to proxy your nmap scans making it very difficult for the scanned network to find the source.

Only scan networks that you are allowed to scan!

First check that you have TOR installed. It should be listening on a local network port tcp/9050.

tcp        0      0 127.0.0.1:9050          0.0.0.0:*               LISTEN      xxxx/tor        

You also need to install a package called proxychains that will proxy all the traffic through TOR. Proxychains has its configuration file in /etc/proxychains.conf. When installed, it will add the TOR connection as one of the available proxies./

[ProxyList]
# add proxy here ...
# meanwile
# defaults set to "tor"
socks4 	127.0.0.1 9050

Now, in order to proxy your nmap traffic through TOR, use the nmap command prepended with proxychains. Remember that due to the nature of the TOR network, you should limit your scans to TCP only. Use the sS (SYN) or sT (CONNECT) scan types.

proxychains nmap -sS -PN -n -p 21,443

proxychains nmap -sT -PN -n -p 21,443

UPDATE

A visitor pointed out that nmap breaks out of proxychains.
Further investigation showed that nmap breaks out of proxychains for the SYN (-sS) scan. The CONNECT (-sT) scan is done via proxychains.

Update ddclient

An older script that I use on OpenBSD to update ddclient hostnames. The script should run with minor modifications on Linux.

#!/bin/sh

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin

IPFILE=/root/scripts/ipcheck.ip
INTF=tun0
MAILRCPT="yourmail"
MAILSUBJECT="DDclient update"

IP=`ifconfig $INTF | grep "inet " | grep -v inet6 | awk '{print $2}' |
sed 's/addr://'`

if [ -f $IPFILE ]; then
 OLDIP="`cat $IPFILE`"
fi

if [ "$IP" != "$OLDIP" ]; then
 echo $IP > $IPFILE
 /usr/bin/logger -t ipcheck_ddclient new IP address -- changed to $IP
 /sbin/ifconfig -a | mail -s $MAILSUBJECT $MAILRCPT
 /usr/local/sbin/ddclient -daemon=0 -syslog -use=ip -ip=$IP
fi

If you are unable to get your public IP from a local interface then you can use dyndns.org.

wget -q -O - checkip.dyndns.org|sed -e 's/.*Current IP Address: //' -e 's/<.*$//'

Use Dropbox with encrypted volume for backups

I use Dropbox to have online backups of my files. Dropbox already provides a good set of protection mechanisms (Two-step verification, …). If you need an additional level then Boxcryptor is worth having a look.

Unfortunately Boxcryptor is not available on Linux but it is compatible with encfs. The blog of Boxcryptor has a post describing in details how you can setup encfs on Ubuntu.

The blog post lacks some useful additional details.

Have encfs available for every user

By default only root users are allowed to use encfs. You can allow non-root users to use encfs.

Modify the /etc/fuse.conf file so that the last line “user_allow_other” does NOT have a leading hash. Save and exit. You do not need to reboot.

Add the non-priv user to the group fuse

You can then use encfs:

$ encfs /home/joeuser/encrypted_data /home/joeuser/decrypted -- -o allow_other

Sync files automatically

I sync my files via rsync from crontab. Before running the rsync I verify if the encrypted volume is mounted.

#/bin/bash

if ! mount | grep encfs >/dev/null; then
 echo "ENCFS not mounted"
else
 rsync -artvuc --delete /home/joeuser/files/ /home/joeuser/decrypted/files
fi

Upgrading Redmine 1.3 to 2.2

The documentation of Redmine describes the upgrade process. I upgraded from Redmine 1.3.1 to Redmine 2.2.2. On an older Ubuntu (10.04.4 LTS) the upgrade steps described in the documentation were not sufficient.

Extract the latest redmine file in your web root. I use a symlink pointing redmine to the latest version. This allows you to keep different version and provides an easy way to switch between versions. You then have to copy the different configuration files (database, configuration) from the ‘old’ setup to the ‘new’ setup. Do not copy the settings.yml file.

cp redmine-1.3.1/config/database.yml redmine/config/
cp -r redmine-1.3.1/files redmine/

The first error message is because of bundle.

rake aborted!
no such file to load -- bundler/setup
gem install bundler
bundle install

The bundle command failed because of libmagick problems. It was not a requirement for older Redmine versions but without the package (and the libraries) I couldn’t get the upgrade to continue.

checking for Magick-config... no
Can't install RMagick 2.13.2. Can't find Magick-config in /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin
apt-get install librmagick-ruby
apt-get install libmagick9-dev

Restart the bundle install.

bundle install

Perform the ‘normal’ migration steps.

rake generate_secret_token
rake db:migrate RAILS_ENV=production
rake redmine:plugins:migrate RAILS_ENV=production 
rake tmp:cache:clear
rake tmp:sessions:clear

Restart Apache (and Passenger). The next error was Ruby (browser) complaining that there was no dispatch file.

no such file to load -- dispatcher

To solve this you have to upgrade Passenger manually.

gem install passenger
passenger-install-apache2-module

Change these configuration files

in /etc/apache2/mods-enabled/passenger.load
  LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-3.0.19/ext/apache2/mod_passenger.so

in /etc/apache2/mods-enabled/passenger.conf
  PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-3.0.19
  PassengerRuby /usr/bin/ruby1.8

Restart Apache and Passenger. Log in to Redmine and go to the Administration part and review the roles and configuration.

Have Redmine installed for you

I can help you if you need help installing or maintaining Redmine.

Graph the first 8 bits of the IP of apache logs

Below is a PHP script that will graph the first 8 bits of the visitors source IP in your apache logs. The bigger the circle, the more visitors you had. You’ll need gd support in PHP. The minimal width of the circle is set to 3, the maximum width is 85. You can run the script with

 ./httpstatus_2_png.php /var/log/apache2/access.log 20x > /var/www/localhost/htdocs/20x.png

The first parameter is the log file. The second parameter is either 20x, 40x or 50x. This graphs the 20x, 40x or 50x HTTP response codes.

Example

The code is also available on GitHub (https://github.com/cudeso/tools/blob/master/httpstatus_2_png.php).

#!/usr/bin/php
<?php
/**
 *	Create a graph based on the first 8 bits of an IPv4 address
 *
 *	Inspiration from http://www.seehuhn.de/pages/internet
 *	As seen in HDMoore http://www.youtube.com/watch?v=b-uPh99whw4
 *
 *	@param 	logfile 								the log file to parse
 *	@param	http_response_code			what type of graph to create, based on http response code (20, 40, 50)
 *	@param	ignore_ips							(optional, what IPs to ignore)
 *
 *	@version 20121009
 * 	@author Koen Van Impe <koen.vanimpe@cudeso.be>
 *	@license New BSD : http://www.vanimpe.eu/license
 *
 */

error_reporting(E_ERROR | E_WARNING | E_PARSE);

// Check for the correct numbers of paramters; display usage
if (!($argc == 3 or $argc == 4)) {
	?>
Create a graph based on the first 8 bits of an IPv4 address
  
Usage: 
 php <?php echo $argv[0]; ?> <logfile> <http_response_code> [ignore_ips] > <image>
  logfile	:	full path of the logfile to parse
  http_response_code : what http response code to plot (20x, 40x or 50x)
  [ignore_ips] : list of IPs to ignore (notation: '192,10,172')
  image : image to export to, fe. image.png

Example:
 php <?php echo $argv[0]; ?> access.log 40x '192,172' > image.png
	<?php
	die();
}

// Variable init
$ip_array = array();
$qt_array = array();
$ip_ignore_array = array();
$logfile = $argv[1];
$color_param = (int) $argv[2]; // strip the x from 50x
$ip = 0;

// Define graph settings
$basegraph_x = 800;
$basegraph_y = 800;
$max_width_circle = 85;

$im = ImageCreate($basegraph_x,$basegraph_y);
$background = ImageColorAllocate($im,0xa5,0x9a,0x7e);
$black = ImageColorAllocate($im,0x00,0x00,0x00);
$red = ImageColorAllocate($im,0x99,0x0f,0x06);
$green = ImageColorAllocate($im,0x63,0x99,0x3e);
$blue = ImageColorAllocate($im,0x28,0x4f,0x99);
ImageFilledRectangle($im,0,0,$basegraph_x,$basegraph_y,$background);

if ($color_param == "50") $color = $blue;
elseif ($color_param == "40") $color = $red;
else $color = $green;

// IPs to ignore
if (strlen($argv[3]) > 0) {
	$ip_ignore_array = explode(",", $argv[3]);
}

// Execute the command, save the output, then walk through the output
$color_param_esc = "^".$color_param;
exec("cat " . escapeshellarg($logfile) . " | awk '{ print $9 \" \" $1; }' |grep " . escapeshellarg($color_param_esc), $output);
if (is_array($output) and count($output) > 0) {
	foreach($output as $line) {
		if (strlen($line) > 0) {
			$arr_line = explode(" ", $line);
			
			// Build up the array width future "width" of the circles
			if (!(strpos($arr_line[1], ":") > 0)) {	// ignore IPv6 values
				$ipstr = substr($arr_line[1], 0, strpos($arr_line[1], ".") );
				// check if it's in the ignore list
				if (!(in_array($ipstr, $ip_ignore_array))) $ip_array[$ipstr]["qt"] = $ip_array[$ipstr]["qt"] + 1;							
			}
		}
	}			
}

// Get the max of the array and recalculate the width of the circles
foreach($ip_array as $key => $val) { $qt_array[] = $val["qt"]; }
if (max($qt_array) > $max_width_circle)	$overflow_value = round( max($qt_array) / $max_width_circle);
else $overflow_value = 1;
for($x = 1; $x<= 255; $x++) { $ip_array[$x]["qt"] = round($ip_array[$x]["qt"] / $overflow_value); }

// Walk through the array, set the coordinates and labels
for($y=0;$y<=15;$y++) {
	for($x=0;$x<=15;$x++) {
		$ipstr = (string) $ip;		

		if (strlen($ipstr) == 1)	$x_offset = 3;
		elseif (strlen($ipstr) == 2)	$x_offset = 8;
		else $x_offset = 11;

		$ip_array[$ipstr]["x_offset"] = $x_offset;
		$ip_array[$ipstr]["x"] = 15 + ($x * 50);
		$ip_array[$ipstr]["y"] = 15 + ($y * 50);			

		if (isset($ip_array[$ipstr]["qt"])) $width = (int) $ip_array[$ipstr]["qt"];
		else $width = 0;
		
		ImageArc($im, $ip_array[$ipstr]["x"] + $x_offset , $ip_array[$ipstr]["y"] + 7, $width, $width, 0, 360, $color);
		ImageFill($im, $ip_array[$ipstr]["x"] + $x_offset, $ip_array[$ipstr]["y"] + 7, $color);

		$ip++;	
	}	
}

// Put the labels on the graph
for ($ip = 1; $ip <= 255 ; $ip++) {
	$ipstr = (string) $ip;
	// Print "ip"-label after the Arc, otherwise the label gets ImageFilled
	ImageString($im, 4, (int) $ip_array[$ipstr]["x"], (int) $ip_array[$ipstr]["y"], $ipstr, $black);
}	
$title = date("Ymd H:i") . " / " . $argv[2];
ImageString($im, 2, 3, 0,  $title, $black);

// Export the image
Header('Content-Type: image/png');
ImagePNG($im);

?>

Run postfix as a blackhole mail server (open relay)

If you want to run Postfix as a blackhole mail server (accepting connection for every user on every domain) then all you have to do is add these settings to /etc/postfix/main.cf :

relayhost = 
relay_transport = relay
relay_domains = static:ALL
smtpd_end_of_data_restrictions = check_client_access static:discard

Note: Postfix is not acting as an open relay in this configuration because it discards mails and is not relaying them to the final destination.

Ubuntu numeric keypad

After the upgrade to Ubuntu 9 I was unable to use the numeric keypad on my external keyboard that’s connected to my laptop. I’ve switched every possible option in the keyboard preferences on and off and found out that you have to disable the Pointer can be controlled using the keypad option. That’s under System, Preferences, Keyboard and Mouse Keys.

Upgrading Ubuntu and no longer access to my encrypted home partition

I recently upgraded my laptop to the latest Ubuntu 8. The upgrade process was easy and quick however after the reboot I was unable to access my home-directory. I have a separate partition on which /home is mounted. This partition is encrypted with cryptsetup.

When I start the cryptdisk daemon it returns an error:

/etc/init.d/cryptdisks start
* Starting remaining cryptodisks...
mount: special device /dev/mapper/cryptohome does not exist
Enter passphrase:


Checking the status with cryptdisk doesn’t show any errors:
/dev/mapper/cryptohome is active:
cipher: aes-cbc-plain
keysize: 256 bits
device: /dev/sda3
offset: 0 sectors
size: 40949685 sectors
mode: read/write

And the device is there:
ls -l /dev/mapper/
total 0
crw-rw---- 1 root root 10, 63 2008-07-12 17:20 control
brw-rw---- 1 root disk 254, 0 2008-07-12 17:47 cryptohome


Crypttab is also configured correctly:

#
cryptohome /dev/sda3 none vol_id


Mounting the /home partition on /dev/mapper/cryptohome (as root) returns :

mount: wrong fs type, bad option, bad superblock on /dev/mapper/cryptohome,
missing codepage or other error
In some cases useful info is found in syslog - try
dmesg | tail or so

with these errors in dmesg
[ 1966.915329] VFS: Can't find ext3 filesystem on dev dm-0

Creating new encrypted partitions (with similar settings) works like a charm. The partition worked perfectly for months before the upgrade and currently I don’t have any clue on how to access my encrypted data. Any help would be highly appreciated.

UPDATE

The problem was not tied to cryptsetup but to something entirely different. My passphrase contains a number of special characters. My keyboard layout was changed to something without the “µ ù ç^” characters. I didn’t notice this during the test because then I was using a rather simple passphrase.

Upgrading Ubuntu and no longer access to my encrypted home partition

I recently upgraded my laptop to the latest Ubuntu 8. The upgrade process was easy and quick however after the reboot I was unable to access my home-directory. I have a separate partition on which /home is mounted. This partition is encrypted with cryptsetup.

When I start the cryptdisk daemon it returns an error:

/etc/init.d/cryptdisks start
* Starting remaining cryptodisks...
mount: special device /dev/mapper/cryptohome does not exist
Enter passphrase:


Checking the status with cryptdisk doesn’t show any errors:
/dev/mapper/cryptohome is active:
cipher: aes-cbc-plain
keysize: 256 bits
device: /dev/sda3
offset: 0 sectors
size: 40949685 sectors
mode: read/write

And the device is there:
ls -l /dev/mapper/
total 0
crw-rw---- 1 root root 10, 63 2008-07-12 17:20 control
brw-rw---- 1 root disk 254, 0 2008-07-12 17:47 cryptohome


Crypttab is also configured correctly:

#
cryptohome /dev/sda3 none vol_id


Mounting the /home partition on /dev/mapper/cryptohome (as root) returns :

mount: wrong fs type, bad option, bad superblock on /dev/mapper/cryptohome,
missing codepage or other error
In some cases useful info is found in syslog - try
dmesg | tail or so


with these errors in dmesg
[ 1966.915329] VFS: Can't find ext3 filesystem on dev dm-0

Creating new encrypted partitions (with similar settings) works like a charm. The partition worked perfectly for months before the upgrade and currently I don’t have any clue on how to access my encrypted data. Any help would be highly appreciated.