Category Archives: Debian

Convert a physical server to a virtual one

Introduction

I needed to “convert” a physical server to a virtual one, in this case I couldn’t and didn’t want to make the same disk partitioning size, but if that’s your case  you can still read. To be more specific, by convert I mean, not to reinstall every services on a brand new virtual machine but simply run what’s on the physical machine. Two reasons, my laziness and most importantly the fact that I’m not sure what was running on. Here goes one solution to do it, we’ll follow these steps:

  • full backup of the physical server
  • prepare the new virtual server on VirtualBox or what you prefer (KVM, etc …)
  • import the backup inside our virtual server’s drive
  • chroot using sytemrescue to update the MBR
  • boot our virtual server
  • Adapt some configuration

Backup !

First things first, do a full backup of your physical server, you can do it that way (adapt it to your use case):

tar -zcvpf mybackup.tar.gz --exclude=/archive --exclude=/mnt --exclude=/proc --exclude=/lost+found --exclude=/dev --exclude=/sys --exclude=/tmp /
Full backup of your server

Be careful with the owners and rights of your files !

Keep that backup somewhere close, we’ll reimport it later.

Virtual Server’s configuration

I’ll only explain how to do it on VirtualBox, if you don’t have it running yet, install it and start it.

Create your new virtual server by clicking on “New”, set your name, your OS type and the version. Just set the parameters as you wish, it depends on your physical server on what you want. Though, once you reach the configuration for the disk, choose vdi and make it a fixed size, it’s quite important for later.

Now your virtual server’s “hardware” configuration is almost done, just set some other settings to your convenience such as the network, boot order (for our test it’s best to select CD first) etc …

We’ll need systemrescue, so download it. Once you have the iso, select it in VirtualBox for your virtual server to boot on. Start it !

This step is about partitioning, so be sure of what you want and what you do. You can simply recreate the same partitions you have on your physical server or just set new ones, just be sure to put enough space for your data to fit in.

qemu-utils and our data

To reimport our data inside the vdi disk we have in VirtualBox we’ll use a cool tool inside qemu-utils. Poweroff your virtual server, it’s very important to not ruin your vdi. Let’s install it and see what we can do with it:

aptitude install qemu-utils
modprod nbd
Install qemu-utils

We need to activate the kernel module nbd too.

Now we’ll link a device /dev/nbd0 to our vdi disk:

qemu-nbd -c /dev/nbd0 <path to your vdi disk>.vdi
Link vdi disk to /dev/nbd0

Create mountpoint directories and mount your partitions:

mount /dev/nbd0p1 /mnt/vs1/boot/
mount /dev/nbd0p3 /mnt/vs1/root/
Mount your partitions

You can now rsync your backup data inside your partitions !

Once your done let’s clean up:

umount /mnt/vs1/root
umount /mnt/vs1/boot
qemu-nbd -d /dev/nbd0
Clean up

Chroot time

Our data is now inside our vdi disk, great, but we can’t boot directly because our MBR doesn’t exist. We need to fix that. Reboot your virtual server on systemrescue. Once you have the prompt you’ll chroot:

mkdir -p /mnt/root/boot
mount /dev/sda3 /mnt/root
mount /dev/sda1 /mnt/root/boot
mkdir /mnt/root/{sys,proc,dev,run,tmp}
mount --bind /proc /mnt/root/proc
mount --bind /sys /mnt/root/sys
mount --bind /dev /mnt/root/dev
mount --bind /run /mnt/root/run
mount --bind /tmp /mnt/root/tmp
chroot /mnt/root /bin/bash
Chroot

If everything went well, you should be “inside” your physical server (as if), check a few files you know to be sure it’s well chrooted (/etc/hosts, content of /home, etc ..) if it’s alright, we can now update the MBR:

update-grub
grub-install /dev/sda
update the MBR

If the commands didn’t yell at you, it’s looking pretty good, you can now poweroff your virtual server. Remove the systemrescue iso and boot your virtual server on its vdi disk, it should work 🙂

Adapt your configurations

A few examples about problems I ran into:

  • careful with your /etc/fstab if you used UUID, you’ll have to change your fstab with a chroot
  • if you didn’t keep the correct owners and rights of your data, it’s possible the virtual server will boot and couldn’t access the data, it will probably try to use a wrong user because of the suid
  • probably a part of your network configuration won’t work, or worse your access to the virtual server, you can clean your firewall with a chroot and then set a NAT rule in VirtualBox to access the machine. In my case, since I had only SSH to my physical server, the keyboard layout was US, and since I have an azerty keyboard it was a pain in the ass. I couldn’t edit easily any file. I decided to SSH on the virtual machine using a NAT rule and to install later the correct layout.

xbox360 controllers with xboxdrv/Dolphin

Ahoy,

I was given a new xbox360 wireless controller at xmas, and I wanted to run my both controllers with one receiver in Dolphin emulator, for whatever reason the normal way didn’t work. Never was the second controller seen in Dolphin !

I went through testing and reading tons of stuff and I ended UP using xboxdrv.

xboxdrv

As mentionned in the man, xboxdrv – A Xbox/Xbox360 gamepad driver that works in userspace !

First step, install the necessary packages:

# apt-get install evtest xboxdrv
Necessary packages

Alright evtest is not that important, but to debug it can be useful.

Now we’ll blacklist xpad to be sure it won’t bother our configuration.

# echo "blacklist xpad" >>/etc/modprobe.d/blacklist.conf
Blacklist xpad

Now you have to reboot before to go on, it’s important.

Back online, plug in your USB wireless receiver. Check it’s there:

# lsusb  | grep -i --color "xbox"
Check the USB wireless receiver is detected

Do the procedure to register your controllers (one after another):

  • Press the X button on the controller
  • Push the “wavy” button on the receiver, it should blink
  • Do the same with the “wavy” button on the controller (tiny little button on top of the controller)
  • The controller should blink briefly
  • Both the receiver and the controller are now linked, light is on, no blinking
  • Do it again for the other controllers

Now we’ll check that xboxdrv is working by listing our receiver(s)/controller(s):

# xboxdrv -L
 id | wid | idVendor | idProduct | Name
----+-----+----------+-----------+--------------------------------------
  0 |   0 |   0x045e |    0x0719 | Microsoft Xbox 360 Wireless Controller (PC) (Port: 0)
  0 |   1 |   0x045e |    0x0719 | Microsoft Xbox 360 Wireless Controller (PC) (Port: 1)
  0 |   2 |   0x045e |    0x0719 | Microsoft Xbox 360 Wireless Controller (PC) (Port: 2)
  0 |   3 |   0x045e |    0x0719 | Microsoft Xbox 360 Wireless Controller (PC) (Port: 3)
xboxdrv -L

As you might have noticed, I have one xbox receiver that can manage up to four controllers (they all do that), explanations:

  • id : receiver’s id (here it’s id 0)
  • wid : controller’s id (column wid from 0 to 3)
  • what’s left doesn’t matter

We can now setup our controllers, you can use my script below, even though you don’t have four controllers, you can still use that script as it is:

#!/usr/bin/env bash

# apt install xboxdrv
# added at session opening
# visudo -f /etc/sudoers.d/xbox360
# floreo ALL = (root) NOPASSWD: /usr/bin/xboxdrv

sudo xboxdrv --daemon  --dbus session \
-i 0 --wid 0 --detach-kernel-driver --mimic-xpad \
--next-controller -i 0 --wid 1 --detach-kernel-driver --mimic-xpad \
--next-controller -i 0 --wid 2 --detach-kernel-driver --mimic-xpad \
--next-controller -i 0 --wid 3 --detach-kernel-driver --mimic-xpad &>/dev/null &
xbox360.sh view raw

As stated in the commentaries, make sure to configure sudo for your user (here mine’s floreo) and the command xboxdrv, indeed it needs to be root to work.

In my case, everytime my session starts, the script does too. Depending on your OS I don’t know where you can put it. You can also do an init script but it’s more of a bother.

UPDATE: Following this post, I’ve added the –mimic-xpad flag in my script so my right stick would eventually work ! It allows you to control the camera in games for example in Steam which is quite better now.

Launch the script:

# bash xbox360.sh
Launch the script

Dolphin emulator

Now open up Dolphin and click on Controllers, in device you should now see evdev/0/Xbox Gamepad (userspace driver) and there should be four of them, the last three ones with a #X. (X being an integer)

Select the first one, press refresh and wait a second. It should now be recognized, press buttons on your controller and the interface should blink in red for every input (try A/B/X/Y first).

If it’s not working, go to troubleshooting else you just can start playing !

Troubleshooting

If nothing works, check again that everything is plugged and that your controllers are registered, do it one more time to be sure.

Kill the script as root and open up two terminals, in one you’ll launch that command:

# xboxdrv --daemon  --dbus session -i 0 --wid 0 --detach-kernel-driver
Start one controller

In the other do that:

# dmesg | grep -iE 'xbox.+'
Check via dmesg

Some result should popup if none then it’s too bad, it certainly mean you didn’t blacklist well xpad, that your kernel’s too old, or something else.

You might try to check with evtest as root:

# evtest
evtest

It should show you a list of what’s connected to your computer, with any luck you could see Xbox Gamepad, if not you’ll have to try to test them.

 

# ls /dev/input/ | grep event*
List your inputs

You should have a list of several inputs, you can try them one after the other like that:

# evtest /dev/input/eventX
Try the inputs one by one

Press buttons on your controller, if it displays anything then it works.

Well voilà, that’s all I can say for help, good luck, it’s quite long to debug that.

How to install Haproxy

Let’s talk about Haproxy which is quite a good reverse proxy, I assume if you are here you know what it is ^^

Let’s walk trough installing it on Raspbian (would work on any *nux), if you are on Debian/Ubunt you can use a cool link that will give you the sources.list.

I wanted to install haproxy using the package manager on my system, but it’s always old versions:

# apt-cache policy haproxy
haproxy:
  Installé : (aucun)
  Candidat : 1.5.8-3+deb8u2
 Table de version :
     1.5.8-3+deb8u2 0
        500 http://mirrordirector.raspbian.org/raspbian/ jessie/main armhf Packages
        100 /var/lib/dpkg/status
Old package version

If the version shown in your package manager if what you want, just install it and you go to the configuration.

I needed the version 1.6, so let’s compile ! If you need an other version, keep reading it’s not that hard.

# replace the version by yours
_HAPROXY_VERSION="1.6.11"
cd /usr/src
wget http://www.haproxy.org/download/${_HAPROXY_VERSION%.*}/src/haproxy-${_HAPROXY_VERSION}.tar.gz &&
cd haproxy-${_HAPROXY_VERSION}/
Download haproxy's sources

Now a few dependencies that you might need too:

apt-get install libpcre3-dev libssl-dev
haproxy dependencies

Finally let’s do it:

make TARGET=custom CPU=native USE_PCRE=1 USE_LIBCRYPT=1 USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_OPENSSL=1
Compile haproxy

As you might notice I require PCRE, LIBCRYPT and most important for me OPENSSL.

If you have a few errors try to google them, don’t forget to do what’s below before you try to compile again:

make clean
Clean the previous try

Now we got the binary that we can move:

cp -a haproxy /usr/sbin/haproxy
Move the binary

Last but not least, you need an init script, there you go (I kindly copied it from a package installed version, all the credit goes to the author)

#!/bin/sh
### BEGIN INIT INFO
# Provides:          haproxy
# Required-Start:    $local_fs $network $remote_fs $syslog
# Required-Stop:     $local_fs $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: fast and reliable load balancing reverse proxy
# Description:       This file should be used to start and stop haproxy.
### END INIT INFO

# Author: Arnaud Cornet <acornet@debian.org>

PATH=/sbin:/usr/sbin:/bin:/usr/bin
PIDFILE=/var/run/haproxy.pid
CONFIG=/etc/haproxy/haproxy.cfg
HAPROXY=/usr/sbin/haproxy
RUNDIR=/run/haproxy
EXTRAOPTS=

test -x $HAPROXY || exit 0

if [ -e /etc/default/haproxy ]; then
	. /etc/default/haproxy
fi

test -f "$CONFIG" || exit 0

[ -f /etc/default/rcS ] && . /etc/default/rcS
. /lib/lsb/init-functions


check_haproxy_config()
{
	$HAPROXY -c -f "$CONFIG" >/dev/null
	if [ $? -eq 1 ]; then
		log_end_msg 1
		exit 1
	fi
}

haproxy_start()
{
	[ -d "$RUNDIR" ] || mkdir "$RUNDIR"
	chown haproxy:haproxy "$RUNDIR"
	chmod 2775 "$RUNDIR"

	check_haproxy_config

	start-stop-daemon --quiet --oknodo --start --pidfile "$PIDFILE" \
		--exec $HAPROXY -- -f "$CONFIG" -D -p "$PIDFILE" \
		$EXTRAOPTS || return 2
	return 0
}

haproxy_stop()
{
	if [ ! -f $PIDFILE ] ; then
		# This is a success according to LSB
		return 0
	fi

	ret=0
	for pid in $(cat $PIDFILE); do
		start-stop-daemon --quiet --oknodo --stop \
			--retry 5 --pid $pid --exec $HAPROXY || ret=$?
	done

	[ $ret -eq 0 ] && rm -f $PIDFILE

	return $ret
}

haproxy_reload()
{
	check_haproxy_config

	$HAPROXY -f "$CONFIG" -p $PIDFILE -D $EXTRAOPTS -sf $(cat $PIDFILE) \
		|| return 2
	return 0
}

haproxy_status()
{
	if [ ! -f $PIDFILE ] ; then
		# program not running
		return 3
	fi

	for pid in $(cat $PIDFILE) ; do
		if ! ps --no-headers p "$pid" | grep haproxy > /dev/null ; then
			# program running, bogus pidfile
			return 1
		fi
	done

	return 0
}


case "$1" in
start)
	log_daemon_msg "Starting haproxy" "haproxy"
	haproxy_start
	ret=$?
	case "$ret" in
	0)
		log_end_msg 0
		;;
	1)
		log_end_msg 1
		echo "pid file '$PIDFILE' found, haproxy not started."
		;;
	2)
		log_end_msg 1
		;;
	esac
	exit $ret
	;;
stop)
	log_daemon_msg "Stopping haproxy" "haproxy"
	haproxy_stop
	ret=$?
	case "$ret" in
	0|1)
		log_end_msg 0
		;;
	2)
		log_end_msg 1
		;;
	esac
	exit $ret
	;;
reload|force-reload)
	log_daemon_msg "Reloading haproxy" "haproxy"
	haproxy_reload
	ret=$?
	case "$ret" in
	0|1)
		log_end_msg 0
		;;
	2)
		log_end_msg 1
		;;
	esac
	exit $ret
	;;
restart)
	log_daemon_msg "Restarting haproxy" "haproxy"
	haproxy_stop
	haproxy_start
	ret=$?
	case "$ret" in
	0)
		log_end_msg 0
		;;
	1)
		log_end_msg 1
		;;
	2)
		log_end_msg 1
		;;
	esac
	exit $ret
	;;
status)
	haproxy_status
	ret=$?
	case "$ret" in
	0)
		echo "haproxy is running."
		;;
	1)
		echo "haproxy dead, but $PIDFILE exists."
		;;
	*)
		echo "haproxy not running."
		;;
	esac
	exit $ret
	;;
*)
	echo "Usage: /etc/init.d/haproxy {start|stop|reload|restart|status}"
	exit 2
	;;
esac

:
/etc/init.d/haproxy

If needed, make some changes ! In my case, everything ran smoothly.

The installation is done, next step is the configuration.