Difference between revisions of "Backups"

From Nuclear Physics Group Documentation Pages
Jump to navigationJump to search
Line 13: Line 13:
 
For old backups in the new format, consolidation works by putting all the data from each backup session into one place, overwriting with the newest data. Nobody's going to look for a specific version of a file they had in 2004 that only existed for three days, so this method is relatively safe, in terms of data retention.
 
For old backups in the new format, consolidation works by putting all the data from each backup session into one place, overwriting with the newest data. Nobody's going to look for a specific version of a file they had in 2004 that only existed for three days, so this method is relatively safe, in terms of data retention.
  
The script that does the backing-up is ''/usr/local/bin/rsync_backup.pl'' and the script that periodically runs it and sends out a notification e-mail is ''/etc/cron.daily/rsynch_backup''.  ''rsync_backup.pl'' determines what disk to put the backup onto, etc.
+
The script that does the backing-up is ''/usr/local/bin/rsync_backup.pl'' and the script that periodically runs it and sends out a notification e-mail is ''/etc/cron.daily/rsynch_backup''.  ''[[rsync_backup.pl]]'' determines what disk to put the backup onto, etc.
  
 
Here [http://www.mikerubel.org/computers/rsync_snapshots/] is a nice little guide on incremental, hardlinked backups via rsync. He sets up some nice little tricks with NFS mounts so that users can access their stuff read-only for any backup that's actually stored. We should do this.
 
Here [http://www.mikerubel.org/computers/rsync_snapshots/] is a nice little guide on incremental, hardlinked backups via rsync. He sets up some nice little tricks with NFS mounts so that users can access their stuff read-only for any backup that's actually stored. We should do this.

Revision as of 13:59, 25 June 2007

NPG backups runs from the dedicated backup server: Lentil, which has 4 hot-swappable drive bays, generally containing SATA drives.

To put in a new (fresh drive):

  1. Locate the oldest disk.
  2. Make sure it is not mounted.
  3. Open the appropriate drive door and slide out drive.
  4. Put new drive in. (there are 4 screws holding the drive in place).
  5. Slide it back in. Take note which Linux drive it registers as: /dev/sdb or /dev/sdc or /dev/sdd or /dev/sde
    NOTE: The order does NOT correspond with the slots, and this order can change after a reboot!
  6. Run /usr/local/bin/format_archive_disk.pl <disk no> <device>
    I.E: /usr/local/bin/format_archive_disk.p 29 /dev/sde[1]
  7. Check that the drive is available: ls /mnt/npg-daily/29

Current Backup System

Newer backups do something involving hard-linking files that haven't changed between backup sessions. Seems like a good idea, but we need to learn exactly how it works. For old backups in the new format, consolidation works by putting all the data from each backup session into one place, overwriting with the newest data. Nobody's going to look for a specific version of a file they had in 2004 that only existed for three days, so this method is relatively safe, in terms of data retention.

The script that does the backing-up is /usr/local/bin/rsync_backup.pl and the script that periodically runs it and sends out a notification e-mail is /etc/cron.daily/rsynch_backup. rsync_backup.pl determines what disk to put the backup onto, etc.

Here [2] is a nice little guide on incremental, hardlinked backups via rsync. He sets up some nice little tricks with NFS mounts so that users can access their stuff read-only for any backup that's actually stored. We should do this.

Legacy Backups

The really old amanda-style backups are tar'ed, gzip'ed, and have weird header info. Looking at the head of them gives instructions for extraction. A script was written to extract and consolidate these backups. Shrinks hundreds of gigs down to tens of gigs, and zipping that shrinks it further. Very handy for OLD files we're not going to look at ever again.