Yet another geeky question

[annathepiper]

This one is for the Linux geeks among you! Here is my situation: I have three Linux servers I want to back up. I want to be able to make tarballs and slap them onto a Windows box share, and from there, dump them down to the tape drive. Right now this process is entirely manual, but I want to be able to automate it because I keep forgetting to make the stupid tarballs and it takes a long time to do it, too.

My question is, how best to automate making the tarballs. I am aware of the backup system Amanda, but I don't know if that's suitable for my needs, nor do I know how to use it. Recommends for or against in this regard are welcome.

In the meantime, what I DO know how to do is set up cron jobs that will make tarballs for me, and since that's nice and easy I was thinking of doing that. Here's the catch, though--I want to be able to name my files in the format machine-backuptype-date.tbz. For example, lodestone-home-9.13.04.tbz to signify the backup of the home partition on lodestone done on 9/13/04. I do know how to just make a little script that would generate the tarball, but what I don't know is how to get it to pull the value in for the date. I.e., whether there is some sort of useful little environmental substitute variable I could use in the script to make it pull the month, day, and year into the name of the file I want to generate.

I can also use this to automate using smbclient to copy the files over onto the Windows server, but again, same question re: pulling the dates into the automation.

("Anna," I fancy I hear some of you cry, "why don't you just use smbclient for generating the tarballs?" A damned good question, one well deserving of an answer. The answer being that smbclient does let you make tarballs, but it doesn't let you gzip or bzip them as well, and I'm doing that to save space on the tapes. Otherwise, I'm pretty sure I'd have to use multiple tapes, which is annoying.

"But Anna!" some of you may also cry, "why don't you just mount the share on the Windows box across Samba so you can just copy the files right over that way?" Another damned good question. The reason I can't do this, and have to use smbclient, is because apparently newmoon and lodestone were both set up without smbfs support in the Linux kernel, and to recompile the kernel to enable this support, sez solarbird, would be a pain in the ass. So smbclient it is!)

Comments

[backrubbear.livejournal.com]

So, making tarballs isn't all that you actually want to do. You actually want management of incremental backups. You don't need a full level 0 backup your entire hard drive or user data every time you want to save your data.

dump/restore, the tools used by Amanda, can be used by hand. However, Amanda is very good at what it does. However, it used to be (I haven't touched it in a number of years) a bit arcane to setup. Once you've learned the tricks, it's not that hard.

The nice thing is that it is reasonably simple to get it to make it's images in a temporary directory. This directory *could* be over the network, but I'd highly recommend letting it dump them in a appropriately sized partition on your disk and then pushing the file over when it's done.

Your next trick is to figure out what to back up and how to back it up. Static files that are unlikely to be changed - very easy. Files that are intended to be open on a regular basis, like db's, mail spools, etc. are a lot more painful. Most db's have their relevant dump facilities - and the trick is to make amanda skip over the files in question and only backup your snapshot files. Mail spools are possibly a bit easier if you avoid doing mbox and simply choose maildir or the qmail equivalent. However, maildir is far more of a pain to deal with if you like grepping your mail.

If any of this sounds intriguing and you feel like mailing me (or just keeping this here), feel free!

[annathepiper]

I took a look at Amanda's web site, and ran screaming from the lack of decent documentation. There is apparently an effort to get the thing documented, but the Docs link that I clicked up there had a big ol' warning at the top that said it was a project in the alpha stages and should not be considered final by any means. Grf.

That said, it sounds like to me that you're in favor of me trying out Amanda. I'll be happy to explore it if it will do what I want it to do--i.e., do backups of three Linux servers onto a Windows share, from which I can then dump stuff to tape. There is a Debian package available for Amanda, so I can quickly install it on our servers.

Regarding where to stick my initial backup files... hrmm. On newmoon, lodestone, and door, I have hard drives of wildly differing sizes and configurations. So what I'm doing manually is just sticking files into a 'backup' directory in my own account space, but excluding that directory when I make the backup of the home partition. That works fine for my purposes, and if Amanda can let me do that too, that'd be cool.

Once the files are created, then yeah, I can use smbclient or whatever to get them over to the Windows box.

[backrubbear.livejournal.com]

I think your primary issue will be wanting to make your backups into the same partition as the partition you're backing up. I'm not quite certain that is possible with Amanda, but it may have changed enough over the years.

That said, many people use Amanda via an ssh connection to another server and it works fine.

[wrog]

How about

date +%m.%d.%y

as in

#!/bin/sh
DATE=`date +%m.%d.%y`
MACHINE=lodestone
TYPE=home
FNAME=${MACHINE}-${TYPE}-${DATE}.tgz
echo $FNAME
 ...

except that since I'm anal, like things to sort nicely, don't want a Y2.1K problem, and prefer not to confuse Windows any more than I have to, I'd probably use

date +%Y-%m-%d

instead, but whatever.

[annathepiper]

Well, if for maximum efficiency I shouldn't back up into the same partition that's being backed up, I can work with that--it's just that previously, I've had the luxury of being able to do backups across a Samba connection and I can't do that now. So this makes it a question of where I can do backups on the various servers that can be good temporary holding space till I move the files over to the actual backup server.

If Amanda can scp or sftp files onto the backup server, that would probably suit me just fine.

[annathepiper]

Thanks Roger! I'll futz around a bit and see if I can make this work. I may wind up learning how to use Amanda, which may make this question moot, but this may help me come up with a good short-term workaround.