Category Archives: ssh

Duplicity to backup safely your data

A good backup tool ? Duplicity

I’ve tried many tools to backup, from bare rsync to tools such as restic or borg. Although they’re all good solutions, my requirements were never fully met. It’s why I chose to test and use duplicity. In this post I’ll focus on full server backup.

Here are my requirements for a good backup tool:

  • compression: I don’t have much to backup but I don’t want to use much either.
  • encryption: I only have a basic FTP that I don’t own, there it’s using GnuPG.
  • incremental backup: Also to save disk space.
  • on a FTP: with restic or borg directly I had lock problems over a curlftpfs mount, any backup would fail, it’s not related to these tools but to my usage.
  • easy to use and restore, it keeps ownership and rights.

Install duplicity

It’s pretty straightforward, install the duplicity package through your package manager on your server. You may also install ncftp if you want to backup over FTP.

# apt-get install duplicity ncftp
Install duplicity and ncftp

Generate your keys

Let’s generate your GNuPG key pair so we can encrypt our backup. This time, on your own computer, and not on your server, do the following.

# gpg --gen-key
Generate your GnuPG key pair

Take a 4096 bits length key, most of the default values should be enough.

# gpg --gen-key
gpg (GnuPG) 1.4.20; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gpg: keyring `/home/floreo/.gnupg/secring.gpg' created
gpg: keyring `/home/floreo/.gnupg/pubring.gpg' created
Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 
Key does not expire at all
Is this correct? (y/N) y

You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"

Real name: Floreo Backup
Email address: xxx@floreo.info
Comment: Floreo Backup
You selected this USER-ID:
    "Floreo Backup (Floreo Backup) <xxx@floreo.info>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
........+++++
.............+++++
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
.+++++
.......+++++
gpg: /home/floreo/.gnupg/trustdb.gpg: trustdb created
gpg: key 1A6F2256 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
pub   4096R/1A6F2256 2017-11-11
      Key fingerprint = CC2D 8FF5 9BB2 8D88 0E30  0056 FC84 663E 1A6F 2256
uid                  Floreo Backup (Floreo Backup) <xxx@floreo.info>
sub   4096R/874F94F5 2017-11-11
Generate your GNuPG key pair

Now export both the private and public keys, the first one you will keep it safely on a USB drive or anywhere safe, it would be used to decrypt your data, the other one will be required on your server. First find out your key UID.

# gpg --list-key
/home/floreo/.gnupg/pubring.gpg
-------------------------------
pub   4096R/1A6F2256 2017-11-11
uid                  Floreo Backup (Floreo Backup) <xxx@floreo.info>
sub   4096R/874F94F5 2017-11-11
List your GnuPG keys

There the UID is 1A6F2256, you can export the keys.

# gpg --output backup_pub.gpg --armor --export 1A6F2256
# gpg --output backup.gpg --armor --export-secret-key 1A6F2256
Export both the pub and private keys

Import your public key

SSH to your server so you can import your key only, first upload the public key through scp for example.

# scp backup_pub.gpg xxx.xxx.xxx.xxx:/home/toto/.
SCP your public key to your server

Your public key being on your server you need to import it to gpg.

# gpg --import backup_pub.gpg
Import your public key.

You would think it’s over but you have to change the trust of the key, do the following command.

# gpg --edit 1A6F2256
Change the trust of your key

One you have the GNuPG prompt, type trust, select optimal, say yes and you’re done !

First server backup

Duplicity allows you to do full backups or incremental ones. Of course the first one is a full backup. One cool thing is that you don’t have to tell it explicitly to do an incremental one once the first one is made.

I suggest you use that very simple script to do your backup, you can edit it to your needs, mostly the exclude part. It backups fully on Monday, following days of the week are incremental. On Monday it also removes old backups, it’s done this dirty way since you cannot use duplicity’s cleanup function without the private key and the passphrase which I refuse to use online. It keeps two weeks of backup which is enough for me. You need to set an autofs mount to erase old backups since I didn’t script the FTP deletion yet.

Now run it, it may take a while so I recommend that you do that in a screen or tmux.

Rather than doing it manually, don’t forget to set a cron task to backup every night your data.

0 2 * * * root bash /root/scripts/backup.sh &>/dev/null
/etc/cron.d/backup

Restore your data

Obviously if you backup anything, you need to restore it some day soon. This time this operation requires the private key so you can decrypt the data ! Do what’s best for you, but it’s safer to keep your private key on your computer, you have to import your private key in gpg first.

# gpg --import backup.gpg 
gpg: key 1A6F2256: secret key imported
gpg: key 1A6F2256: public key "Floreo Backup (Floreo Backup) <xxx@floreo.info>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg:       secret keys read: 1
gpg:   secret keys imported: 1
Import your private key to your local computer

Next step is really simple you have to revert the duplicity command to restore.

# export PASSPHRASE=<PASSPHRASE>
# export FTP_PASSWORD=<FTP PASSWORD>
# duplicity  ftp://<USER>@<HOST>/<REMOTE_DIRECTORY>/ <LOCAL_DIRECTORY>
# unset PASSPHRASE FTP_PASSWORD
Restore your encrypted data locally

Finally rsync your data to your server, I know it looks complicated but it makes sure that your private key is never on your server.

Do not forget to remove your private key from your computer, it’s safer too.

gpg --delete-secret-key 1A6F2256
Delete your imported private key

A word of advice if you have several backup chains, for example I backup fully on Monday and do incremental ones for other days and keep two weeks top, duplicity will show you only the last full backup even if you restore stuff before the last full backup. It’s weird, but your restoration would work, it would just display the wrong date. To be sure I check the last mail.log, you might do it as well.

Check that your backups are good quite often. For further details, you can find some more explanations following theĀ Ubuntu documentation.

How to make SSHFS mount with SSH key and password

Let’s talk today about SSHFS mount, sometimes you just can’t do NFS or CIFS mount just because ^^

So one solution could be to use SSHFS. I won’t argue about speed, security or benchmarking the thing, I just had no other choice than doing SSHFS so here’s a way to do it with an SSH key and with only a password. (yea that’s a terrible idea but again, sometimes you have no choice)

Obviously if you have things to say about that method, drop a comment I’ll be glad.

I decided to show up only the fstab mount, if you need to do it on the fly then just adapt it šŸ™‚

Prelude

Both example will be self explanatory for the most, you just have to change theĀ words that I put uppercase.

uid/gid fields are set for the local server’s user (check /etc/passwd), set it to the proper user.

idmap=user is THE trick to keep the correct uid/gid mapping on both servers !

Before to get started, we will be mountingĀ /home/REMOTE_USER/data/ from the remote server inĀ /mnt/data/ on the local server therefore make sure to mkdir the local directory /mnt/data as your mount point.

SSHFS mount in /etc/fstab with a SSH key

Here it’s the IdentifyFile parameter that’s the most important, it must be the SSH private key, don’t forget to put the SSH pub key on the other server.

sshfs#REMOTE_USER@REMOTE_HOST:/home/REMOTE_USER/data/ /mnt/data/ fuse            IdentityFile=/home/LOCAL_USER/.ssh/THEKEY,uid=LOCAL_UID,gid=LOCAL_GID,users,idmap=user,noatime,allow_other,_netdev,auto_cache,reconnect     0 0
sshfs fstab mount with SSH key

Just do:

# mount -a
Time to mount !

SSHFS mount in /etc/fstab with a password using sshpass

First install sshpass, on debian (for other OS do a research, it shouldn’t be hard):

apt-get update && apt-get install sshpass
Install sshpass on debian

This time the important parameter isĀ ssh_command=/home/LOCAL_USER/passwd.sh it’s just a simple script that will do the trick.

sshfs#REMOTE_USER@REMOTE_HOST:/home/REMOTE_USER/data/ /mnt/data/ fuse            ssh_command=/home/LOCAL_USER/passwd.sh,uid=LOCAL_UID,gid=LOCAL_GID,users,idmap=user,noatime,allow_other,_netdev,auto_cache,reconnect     0 0
sshfs fstab mount with only a password

Make sure now to edit the file for the ssh_command, hereĀ /home/LOCAL_USER/passwd.sh, simply changeĀ REMOTE_PASSWORD to the right password.

#!/bin/bash

sshpass -p REMOTE_PASSWORD ssh $*
Content of /home/LOCAL_USER/passwd.sh

Make it safer !

# chown LOCAL_USER:LOCAL_USER /home/LOCAL_USER/passwd.sh && chmod 700 /home/LOCAL_USER/passwd.sh
Make it safer !

And now it’s time to mount:

# mount -a
Time to mount !

Debugging

Probably it won’t all go well, wether you set the wrong password, the wrong key or whatever, if so just add the following options debug,sshfs_debug into your fstab:

sshfs#REMOTE_USER@REMOTE_HOST:/home/REMOTE_USER/data/ /mnt/data/ fuse            debug,sshfs_debug,ssh_command=/home/user/passwd.sh,uid=LOCAL_UID,gid=LOCAL_GID,users,idmap=user,noatime,allow_other,_netdev,auto_cache,reconnect     0 0
sshfs example with debug options

Conclusion

Well, not much to add, it’s pretty much useful to use SSHFS but it can be tricky, obviously, do prefer the SSH key method.

Kick someone out of the server

Hello,

it happens someone is logged in on a server and you don’t want that or just a “little prank” to a friend or coworker (keep in mind the first example, i’m not responsible).
So at first check out who’s logged in and with which user:

# who
root     pts/0        2015-05-10 16:14 (xxx.xxx.xxx.xxx)
root     pts/3        2015-05-10 19:08 (my-super-reverse)

You noticed two persons are logged in, both as root, I’ll pretend I’m the first one and the second (i.e my-super-reverse) is someone else, and just right now I have to kick him out.
Because I’m well educated, I will tell him first, like that:

# echo 'Have a nice day sir, you are out from now! cya :)' > /dev/pts/3

With this line, you directly write in his TTY. Look on the previous who command, I only took the reference to his TTY (i.e pts/3), ja it’s pretty cool.
And now, find the PID of the TTY of pts/3

# ps -ft pts/3
UID        PID  PPID  C STIME TTY          TIME CMD
root     18999 18997  0 19:08 pts/3    00:00:00 -bash
root     20240 18999  1 19:20 pts/3    00:00:00 watch -n 1 echo toto

In this example, there are two things, first the -bash is the TTY session and secondly, the “watch -n 1 echo toto”. This guy is doing a watch to echo “toto”, OUT!
What matters here is the first PID, the TTY session. We can now kick him out.

kill -9 18999

He’s out! You can do a who once again to check.
Don’t tread on me! šŸ™‚

How to deal with several ssh keys easily

Hello, this post will be about how to use several ssh keys in a “simple” way.

I had the problem of having several ssh keys to connect to different servers and also to use git with a different key.
I know I’m not quite clear with this, so let’s take an example.

You have two keys, one to connect to a server and an other one to use git (github, bitbucket, whatever …).

You want to clone a project using git :

git clone git@git.florianleleu.com:/myAwesomeProject

And you would do this to connect to the server s1.farm.florianleleu.com :

ssh florian@s1.farm.florianleleu.com -p 2222

Your keys are in ~/.ssh/, though which one will be used for the server and which one for git? Stuck.

I hope it’s more clear now. šŸ˜€

Here’s one way to solve this problem.
You have to create this file ~/.ssh/config
Give it some “good” rights (i.e chmod 600 ~/.ssh/config)

Here is an example of what you can write in this file, the explanations will follow :

Host git git.florianleleu.com
    HostName git.florianleleu.com
    User git
    IdentityFile ~/.ssh/git/id_rsa

Host *.farm.florianleleu.com
    User florian
    Port 2222
    IdentityFile ~/.ssh/farm/id_rsa

Options shown :

  • Host => alias or the pattern(s) you want to match, some regex are allowed
  • Hostname => the real host you want to connect to
  • User => the user with which you will be connected on the remote server
  • Port => the port of the remote server (default 22)
  • IdentityFile => the path to your private key

And now how to use it :

git clone git:/myAwesomeProject

In this command: git cloneĀ git:/myAwesomeProject, what’s in bold will be replaced by git@git.florianleleu.com because it is matched by the first Host (i.e alias), and obviously, it will do what was wanted at first, which is to take the keyĀ ~/.ssh/git/id_rsa. Note that you can also write it git clone git@git.florianleleu.com:/myAwesomeProject, but the alias is quite handy.

ssh s1.farm.florianleleu.com

In this command: ssh s1.farm.florianleleu.com, it’s matched by the second Host, and will be replaced by florian@s1.farm.florianleleu.com -p 2222 and use the fileĀ ~/.ssh/farm/id_rsa.

Obviously, not everything is said, there are other options ! RTFM šŸ™‚

man ssh_config