Recovering from a lost connection when upgrading Ubuntu via ssh

Posted on 2011-05-30 by Earl C. Ruby III

I wanted to upgrade my desktop machine at work to the latest version of Ubuntu, but since it takes several hours to upgrade an Ubuntu host, and I have work to do during the day, I figured I could log into my workstation from home in using ssh and start the upgrade remotely.

So I logged into my workstation from home and ran:

> sudo apt-get install update-manager-core
> sudo do-release-upgrade

The upgrade script warned me that I was using ssh and asked if I was sure I wanted to continue. I said “Y”, and a little while later the upgrade manager was busy downloading upgrade packages.

I planned to check it a couple of times that night, answer any package upgrade questions that popped up, and then in the morning when I got to work the upgrade would be complete.

Of course what actually happened was that I got side-tracked onto some other problem that night, forgot about the upgrade in progress, and when I got to work the next day my workstation was in a state of limbo, with the upgrade halfway complete, waiting for me to answer some question on the screen — at my house.

Luckily the Ubuntu developers who created the ssh upgrade process run that upgrade inside of a screen session. As the screen pages states, “Screen is a full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells).”

So at work all I had to do was get the list of current screen sessions:

> sudo screen -list
There are screens on:
        9129.ubuntu-release-upgrade-screen-window       (05/17/2011 08:50:08 PM)        (Attached)
2 Sockets in /var/run/screen/S-root.

Invoke screen using the “-d -r sessionowner/[pid.tty.host]” flags:

> sudo screen -d -r root/9129.ubuntu-release-upgrade-screen-window

… and I could pull up the screen at work that had been displaying at my home. Once I answered the remaining questions about whether to keep my custom configuration files or use the new, packaged configuration files my workstation rebooted and the latest version of Ubuntu booted right up.

Adding an external encrypted drive with LVM to Ubuntu Linux

Posted on 2010-02-10 by Earl C. Ruby III

I recently added an external eSATA drive to my home computer so I could back up critical data from my home network to one drive. I bought a Western Digital 1TB “green” drive and a Thermaltake external hard drive enclosure with eSATA and USB connectors.

Since my internal hard drives are encrypted it didn’t make sense to back up all of that data to an unencrypted external drive. I’d read Uwe Hermann’s excellent how-to article on disk encryption, but he didn’t cover setting up an LVM partition, which I always use so I can change drive volume sizes on the fly.

This is what I did to set up an external encrypted drive with LVM on an Ubuntu system:

Open a terminal
Get a root prompt:
```
sudo /bin/bash
```
Watch the system log:
```
tail -f /var/log/messages
```
Attach the external drive. The system log tells me that it was detected as /dev/sdc.
Check the drive for bad blocks (takes a couple of hours):
```
badblocks -c 10240 -s -w -t random -v /dev/sdc
```
Write random data to the entire drive. This step takes all night, but it ensures that never-written drive space can’t be differentiated from encrypted data if someone ever tries to crack the drive. (If you’re going to do this, you might as well do it right.)
```
shred -v -n 1 /dev/sdc
```

Create one big LVM partition on the drive using fdisk. Set up one big primary partition /dev/sdc1, set the tag to system id “8e” LVM, and write the changes to disk:

> fdisk /dev/sdc                                                                                                                                              
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel                                                                                                  
Building a new DOS disklabel with disk identifier 0xa6846916.                                                                                                                       
Changes will remain in memory only, until you decide to write them.                                                                                                                 
After that, of course, the previous content won't be recoverable.                                                                                                                   


The number of cylinders for this disk is set to 121575.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:               
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs                                                                                                                                 
   (e.g., DOS FDISK, OS/2 FDISK)                                                                                                                                                    
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)                                                                                                      
                                                                                                                                                                                    
Command (m for help): p                                                                                                                                            
                                                                                                                                                                                    
Disk /dev/sdc: 999.9 GB, 999989182464 bytes                                                                                                                                         
255 heads, 63 sectors/track, 121575 cylinders                                                                                                                                       
Units = cylinders of 16065 * 512 = 8225280 bytes                                                                                                                                    
Disk identifier: 0xa6846916                                                                                                                                                         
                                                                                                                                                                                    
   Device Boot      Start         End      Blocks   Id  System                                                                                                                      
                                                                                                                                                                                    
Command (m for help): n                                                                                                                                            
Command action                                                                                                                                                                      
   e   extended                                                                                                                                                                     
   p   primary partition (1-4)                                                                                                                                                      
p                                                                                                                                                                  
Partition number (1-4): 1                                                                                                                                          
First cylinder (1-121575, default 1): [ENTER]                                                                                                                      
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-121575, default 121575): [ENTER]
Using default value 121575

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sdc: 999.9 GB, 999989182464 bytes
255 heads, 63 sectors/track, 121575 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xa6846916

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1      121575   976551156   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Use cryptsetup to encrypt the drive:

cryptsetup --verbose --verify-passphrase luksFormat /dev/sdc1

Unlock the drive:

cryptsetup luksOpen /dev/sdc1 backupexternal

Create the LVM physical volume:
```
pvcreate /dev/mapper/backupexternal
```

Create the LVM volume group:

vgcreate xbackup /dev/mapper/backupexternal

Create a logical volume within the volume group:
```
lvcreate -L 500G -n backupvol /dev/xbackup
```
At this point you have a device named /dev/xbackup/backupvol, so create a filesystem on the logical volume:
```
mkfs.ext4 /dev/xbackup/backupvol
```

Mount the volume:

mount /dev/xbackup/backupvol /mnt/backup

To get the volume to mount automatically at boot time add this line to your /etc/fstab file:
```
/dev/xbackup/backupvol      /mnt/backup     ext4    defaults        0 5
```
To be prompted for the decryption key / passphrase at boot time first get the drive’s UUID:
```
ls -l /dev/disk/by-uuid
```
(In my example I use the UUID for /dev/sdc1)

Then add this line to the /etc/crypttab file:

backupexternal UUID=[the UUID of the drive] none luks

That’s it. You now have an external, encrypted hard drive with LVM installed. You’ve created one 500GB volume that uses half the disk, leaving 500GB free for other volumes, or for expanding the first volume.

Hope you find this useful.

Installing Skype on Ubuntu 9.04 using Synaptic

Posted on 2009-07-13 by Earl C. Ruby III

Want to make free phone calls using Skype? Running Ubuntu 9.04? Want to make sure that Skype stays up-to-date with the latest security patches and updates once you install it? Here’s how:

On the Skype Download for Linux page Skype lists Ubuntu 7.04-8.04 as the only supported versions of Ubuntu. You can install Skype on Ubuntu 9.04, but when new updates or security fixes come out you won’t get an automatic update. If you want to make sure that your Skype installation stays up-to-date you should add skype.com as a valid software repository and install Skype from that. If you do this you’ll automatically get updates when they’re available.

To add skype.com as a repository:

Start up Synaptic Package Manager. (On Gnome go to System > Administration > Synaptic Package Manager)
Select Settings > Repositories
Click the Third-Party Software tab
Click Add
Enter “deb http://download.skype.com/linux/repos/debian/ stable non-free” as the APT line
Click the Reload button in the upper left-hand corner to download the list of available files from download.skype.com
Type “skype” in the Quick Search box and you should see Skype listed as an installable package
Mark it for installation and install

Skype should now appear on your Applications > Internet menu.

Update: At the time that I wrote this the instructions worked. However, it looks as if Skype has removed their apt repositories for 9.04. If you attempt to connect to http://download.skype.com/linux/repos/debian/ you’ll get a “Hi, this page is off limits” error message.

If you are running Ubuntu 10.04 or later you can get Skype directly from Canonical. See https://help.ubuntu.com/community/Skype for more info.

Change your Skype availability status on Ubuntu via cron

Posted on 2008-10-10 by Earl C. Ruby III

I used Skype when it first came out and then stopped because there were only a few other people I knew who used it regularly. Since they were all in my local calling area, if I wanted to talk to any of them for free I could just use a phone.

I just started using it again — for work — to talk with customers and consultants who do not live in the Bay Area. However, I found that my Thursday telecommute days were causing problems: Skype would be running on my work computer and people trying to call me would get ring-no-answer because I was actually at home.

Turns out you can start and stop Skype via cron the same way you can start and stop Pidgin Instant Messenger via cron, you just need to set a few environment variables and it works great.

First create the script ~/bin/export_x_info which looks like this:

#!/bin/bash
# Export the dbus session address on startup so it can be used by cron
touch $HOME/.Xdbus
chmod 600 $HOME/.Xdbus
env | grep DBUS_SESSION_BUS_ADDRESS > $HOME/.Xdbus
echo 'export DBUS_SESSION_BUS_ADDRESS' >> $HOME/.Xdbus
# Export XAUTHORITY value on startup so it can be used by cron
env | grep XAUTHORITY >> $HOME/.Xdbus
echo 'export XAUTHORITY' >> $HOME/.Xdbus

Create this script, type chmod 700 ~/bin/export_x_info so you can execute it, then execute it, then add it to System > Preferences > Sessions > Startup Programs so it will execute every time you start your computer and record the latest session address and XAUTHORITY value.

This script creates a 4-line file ~/.Xdbus with the current session address and XAUTHORITY value. By sourcing this file in the crontab file your scripts can now use dbus to send messages to X-Windows applications. To start and stop Skype at work I added these lines to my work computer’s crontab file:

# Skype on/off
00 09 * * Mon,Tue,Wed,Fri source ~/.Xdbus; /usr/bin/skype &
30 17 * * Mon,Tue,Wed,Fri killall skype

On my home computer I have:

# Skype on/off
00 09 * * Thu source ~/.Xdbus; /usr/bin/skype &
30 17 * * Thu killall skype

So on Thursdays when I telecommute Skype is running at home, the rest of the week it’s running at my office, and at night it’s turned off so customers aren’t calling me in the middle of the night.

When Skype isn’t running other users see your status as “Offline”. When cron restarts Skype it starts up with the same status it had when the process was killed. I just leave my status set to “Online,” so when cron starts Skype it shows everyone that I’m online. When cron kills the Skype process at the end of the day my status changes to “Offline” and people can’t call me.

Hope you find this useful.

Update Pidgin IM status on Ubuntu using cron

Posted on 2008-08-08 by Earl C. Ruby III

At work we use a Jabber instant messenger (IM) server for internal company communications, so that regardless of whether someone is in the office, working from home, or on the road, they can be reached via IM.

I’m running Ubuntu both at work and at home and I use the Pidgin IM client to talk to the Jabber server. I work from home on Thursdays, and I’m always forgetting to turn Pidgin off when I leave work on Wednesday. I usually end up ssh-ing into my work box from home and killing the Pidgin client off remotely, but sometimes I forget and when I come back to work on Friday there are a half-dozen “Are you there?” -type messages on my Pidgin work-client.

So I figured I’d automate the process, automatically setting Pidgin status to “Away” and “Available” using cron, turning the work-client off entirely on Thursdays and weekends, and automatically turning the home-client on Thursday mornings and off Thursday night.

I did a little digging and found a command-line program called purple-remote that allows me to automatically update the Pidgin status and message lines. The purple-remote program is included in the libpurple-bin package, which I installed with System > Administration > Synaptic Package Manager.

Once purple-remote was installed, I fired up a terminal and did a little experimenting on the command line. I found I could set Pidigin’s status to “Away” and the status message to “At lunch” by typing:

/usr/bin/purple-remote "setstatus?status=away&message=At lunch"

I could set the status to “Available” and blank the status message by typing:

/usr/bin/purple-remote "setstatus?status=available&message="

So far so good. I can also cause Pidgin to exit with:

/usr/bin/purple-remote "quit"

Starting Pidgin is just a matter of running:

/usr/bin/pidgin

Time to set up the cron jobs. I fired up crontab -e and entered:

# IM Status
SHELL=/bin/bash
00 08 * * Mon,Fri /usr/bin/pidgin &
01 08 * * Mon,Tue,Wed,Fri /usr/bin/purple-remote "setstatus?status=away&message="
00 09 * * Mon,Tue,Wed,Fri /usr/bin/purple-remote "setstatus?status=available&message="
30 13 * * Mon,Tue,Wed,Fri /usr/bin/purple-remote "setstatus?status=away&message=At lunch"
30 14 * * Mon,Tue,Wed,Fri /usr/bin/purple-remote "setstatus?status=available&message="
30 17 * * Mon,Tue,Wed,Fri /usr/bin/purple-remote "setstatus?status=away&message="
00 19 * * Wed,Fri /usr/bin/purple-remote "quit"

This would start Pidgin at 8:00am, set my status to “Away” at 8:01am, “Available” at 9:00am, “Away – At lunch” at 1:30pm, “Available” again at 2:30pm, “Away” at 5:30pm. On Wednesday and Friday nights at 7:00pm the client shuts down entirely, on Monday and Friday mornings Pidgin gets restarted. That will leave the client off all day on Thursdays and on weekends when I’m not at the office. If my schedule changes for any reason I can still update my status in Pidgin manually.

Looks good, but this doesn’t work. Although the commands listed above work just fine on the command line, they’d fail when they were executed from cron. Checking my mail I found error messages like this:

Traceback (most recent call last):
File "/usr/bin/purple-remote", line 16, in
obj = dbus.SessionBus().get_object("im.pidgin.purple.PurpleService",
 "/im/pidgin/purple/PurpleObject")
File "/var/lib/python-support/python2.5/dbus/_dbus.py", line 218, in __new__
mainloop=mainloop)
File "/var/lib/python-support/python2.5/dbus/_dbus.py", line 107, in __new__
bus = BusConnection.__new__(subclass, bus_type, mainloop=mainloop)
File "/var/lib/python-support/python2.5/dbus/bus.py", line 121, in __new__
bus = cls._new_for_bus(address_or_type, mainloop=mainloop)
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.Spawn.ExecFailed:
 dbus-launch failed to autolaunch D-Bus session: Autolaunch error:
 X11 initialization failed.

The key part of that message is “dbus-launch failed to autolaunch D-Bus session: Autolaunch error: X11 initialization failed.” dbus is a messaging system, and X11 is the implementation of X-Windows Ubuntu uses, which is what the Gnome desktop runs on. Pidgin is an application running on the Gnome desktop / X-Windows. The error message is saying that the program failed to send a message to X11 (X-Windows) and on to Pidgin.

There are three problems here:

The first is that in order for purple-remote to send messages to Pidigin via DBUS it has to work it needs to know the value of the DBUS_SESSION_BUS_ADDRESS environment variable. (Kudos to explicitly ambiguous and his article at http://ubuntuforums.org/showthread.php?t=632580 for pointing me towards this solution.)
In order to start Pidgin at 8:00am Pidgin needs to know the value of the of the XAUTHORITY environment variable otherwise it won’t be authorized to start up on a screen that I’m logged into.
In order to start Pidgin at 8:00am Pidgin needs to know the value of the DISPLAY environment variable so it knows what screen to start up on.

When you start up Ubuntu the dbus daemon starts up and creates a unique session address. Your applications have to know this session address in order to send messages using the daemon. The address is stored in the DBUS_SESSION_BUS_ADDRESS environment variable when you log in, so if you try to run an application from the command line it gets the session address from the DBUS_SESSION_BUS_ADDRESS environment variable. Since cron runs in it’s own environment it doesn’t know the value of DBUS_SESSION_BUS_ADDRESS, so programs that depend on dbus fail when you try to run them from a cron job.

You can see the value by typing:

> env | grep DBUS_SESSION_BUS_ADDRESS
DBUS_SESSION_BUS_ADDRESS=unix:abstract=
/tmp/dbus-AmUs20000,guid=6cce82d52ca190000000000000000000

Likewise, the value of XAUTHORITY changes every time you restart your computer. You can see the current value by typing:

> env | grep  XAUTHORITY

DISPLAY remains the same between reboots. To get the value for your system, type:

> env | grep DISPLAY
DISPLAY=:0.0

Add the “DISPLAY=:0.0” line at the beginning of your crontab file and it’ll be set for Pidgin.

In order to make the environment variables available to cron jobs I created the program ~/bin/export_x_info which looks like this:

#!/bin/bash
# Export the dbus session address on startup so it can be used by cron
touch $HOME/.Xdbus
chmod 600 $HOME/.Xdbus
env | grep DBUS_SESSION_BUS_ADDRESS > $HOME/.Xdbus
echo 'export DBUS_SESSION_BUS_ADDRESS' >> $HOME/.Xdbus
# Export XAUTHORITY value on startup so it can be used by cron
env | grep XAUTHORITY >> $HOME/.Xdbus
echo 'export XAUTHORITY' >> $HOME/.Xdbus

SHELL=/bin/bash
DISPLAY=:0.0
# IM Status
00 08 * * Mon,Fri source ~/.Xdbus; /usr/bin/pidgin &
01 08 * * Mon,Tue,Wed,Fri source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=away&message="
00 09 * * Mon,Tue,Wed,Fri source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=available&message="
30 13 * * Mon,Tue,Wed,Fri source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=away&message=At lunch"
30 14 * * Mon,Tue,Wed,Fri source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=available&message="
30 17 * * Mon,Tue,Wed,Fri source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=away&message="
00 19 * * Wed,Fri source ~/.Xdbus; /usr/bin/purple-remote "quit"

The call to source ~/.Xdbus on each line loads the DBUS_SESSION_BUS_ADDRESS and XAUTHORITY environment variables before executing the purple-remote command or starting Pidgin.

Now the cron works and my Pidgin status is constantly updated. The cron on my home computer is much simpler:

SHELL=/bin/bash
DISPLAY=:0.0
# IM Status
00 08 * * Thu source ~/.Xdbus; /usr/bin/pidgin &
01 08 * * Thu source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=away&message="
00 09 * * Thu source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=available&message="
30 13 * * Thu source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=away&message=At lunch"
30 14 * * Thu source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=available&message="
30 17 * * Thu source ~/.Xdbus; /usr/bin/purple-remote "setstatus?status=away&message="
00 19 * * Thu source ~/.Xdbus; /usr/bin/purple-remote "quit"

The same technique works for other X-Windows programs as well. For instance, I added this line to my cron:

30 17 * * Mon,Tue,Wed,Fri source ~/.Xdbus; /usr/bin/notify-send "Go Home" "Time to Go"

So every day at 5:30pm a “notify” message pops up reminding me that it’s time to go home.

Hope you find this useful.