lundi 18 décembre 2017

urBackup episode 3

  • Windows backup ended fine.  Other incremental backups have followed during the few times I boot this laptop under Windows 10 instead of Linux.  So far so good.  I've also tested a restore (one JPEG file), that went fine as well.
  • I need to do something to save the config of the server itself I guess.  BackupPC provided the localhost host by default to do that, but the urBackup server doesn't have a default client on the server itself apparently; will check if I can install a Linux client on the same box and make that happen.
  • The rtcwake script seems to work fine on kid2 too.  I enhanced it a bit to check for loadavg because it should not shutdown while the load average is anywhere above idle; so i don't need to do specific checks about urBackup processes and so forth
  • One day I'll have to look at the email configuration too.  That's definitely broken by default for a home network
  • The kid2 hardware looks like a good idea to start migrating to urBackup while not removing BackupPC immediately.  The only problem is I will need one more "big" hard disk before I can add any more clients to urBackup.

mercredi 13 décembre 2017

xscreensaver slide show with Flickr image stream

OK, pretty simple to do actually. The most difficult part was to find out what my Flickr ID was again.
luc@kid2:~$ grep imageD .xscreensaver 
imageDirectory: https://api.flickr.com/services/feeds/photos_public.gne?id=125435399@N06

mardi 12 décembre 2017

urBackup episode 2

I need more space for urBackup and no space for /home (I want to get rid of this home directory anyway). So unmounted /home, remounted it under /mnt, tried to move everything on the /home (in root filesystem), noticed that there wasn't enough room there because of all the cache and trash in there, did some cleanup then moved the rest... ok, I also moved the urBackup data on this 300Gb hard disk. Then changed /etc/fstab, remounted the disk under it's new location, restarted the urbackupsrv service... got some permission denied... changed the owner:group on the mount point, restarted the service again... that appears to have worked. I left the backup for my test Windows client running. It's quite big (lots of photos and videos) but hopefully there will be enough room this time.

Puppylinux on the Hercules EC-800-H20G/S

Rebooted this old little fellow. The crunchbang linux was still happily running. No real update path I guess (crunchbang distribution has been replaced by something else apparently) so why not trying puppylinux on it. First attempt failed (I had used the same SD card as for kid2... except that the eCafé has a 32bit processor apparently not a 64bit one). Second attempt, after downloading rufus (to create the bootable SD partition) and a proper 32bit Puppylinux image, was OK. I did a full install, but got stuck in the wifi activation. Even after remembering to use Ctrl+F1 to activate the wifi interface, I couldn't connect on my WPS access point. Will have to try harder, maybe on another AP, somewhat later. In the meantime puppylinux seems happy on this old hardware. I'll have to check how to reduce the X resolution to match the hardware panel which is rather limited on this eCafé. And/Or hook it up to an external monitor and hoping to get a little bit more pixels that way.

urBackup proof of concept

So far I had not tested this solution because it wasn't available by default in the Ubuntu repository. However after my problems with RsyncD on Windows 10, I figured out that maybe another backup solution should be (at least) investigated, just in case backuppc wouldn't do well with Windows backups. I didn't want to go back to SMB backups if I could avoid it. urBackup looks like a nice, open source, polished solution. It is client-based (e.g. a client package has to be installed on each machine to be backed up) but at least it seems to support Windows very well (and I still have to test the Linux client which is essential for my needs). It promises to support backup compression and de-duplication although it is not clear what level of support (at least for compression it relies on the underlying OS and filesystem. Ext4 is not doing that AFAIK). So after adding a PPA, I was able to install a server for testing on kid2. The server configuration was pretty easy (email won't work but that's not urBackup's fault). I downloaded a client on run12l and set it up real quick, then the first backup started. I had to stop it before it could complete, will need to come back to it later.

NFS/SSH/RSYNC issues linked to DNS / DHCP

The NFS & Backuppc server was using DHCP but with a fixed IP configured in the DHCP server (in the router). Sometimes this box was unable to resolve names of other machines on the LAN. I tried fidling with /etc/nsswitch.conf without success. I decided to move from DHCP to fixed IP since the server IP is fixed already anyways; So I also removed NetworkManager* and used /etc/network/interfaces to setup the eth0 interface. I also re-enabled the cronjob to suspend (shut down) the server when idle. Then on the next day I noticed a reboot (clock-driven) and tried to login after a couple minutes but the box wasn't responsive... I connected a screen and keyboard to troubleshoot the issue, just to discover that I forgot to add "auto eth0" in my new /etc/network/interfaces configuration. So after each reboot the server stayed offline :-\ Easy fix, now the hourly suspend/reboot seems to work fine. The script doesn't reboot in case a backup is in progress, or NFS clients are connected. To be verified after some more days.

Backuppc/RsyncD unresponsive on Windows 10

One of my windows backuppc clients is hanging during backups. On the status screen, it shows as "backup in progress" but no transfer PID is shown, and it takes ages in that state. Usually the user shuts down before the backup completes (and the status seems to stay the same). I upgraded RsyncD from 3.0.9 to 3.1.x, applied Windows Updates and rebooted. The next backup seems to have worked (in 80 minutes!) but has been interrupted again. To be checked. It might also be linked to DNS or interface problems (I'm currently migrating the backuppc server from a DHCP to a fixed IP configuration).

samedi 9 décembre 2017

Multi-Boot Ubuntu/Puppylinux/Windows with Grub4Dos

Grub4Dos is still new to me.  Here are the entries that work in menu.lst to do what I want.  Except that I couldn't find a way to let the Puppylinux find its root file system by UUID, so that entry is using a device name (/dev/sde1 in my case).




# menu.lst produced by grub4dosconfig-v1.9.2
color white/blue black/cyan white/black cyan/black
#splashimage=/splash.xpm
timeout 10
default 0
# Full installed Linux
title Xenialpup64 7.5 Full install (root on sde1)
  uuid abf7b3c6-246c-4189-82a2-a8d41784ba8d
  kernel /boot/vmlinuz root=/dev/sde1 ro
# Ubuntu on another drive, note that the root=PARTUUID works
title Ubuntu 17.10 (internal SATA drive by UUIDs)
  uuid ca3a2c98-fce2-4d11-bb23-995d1ef1c7d7
  kernel /vmlinuz root=PARTUUID="676b6100-04" ro
  initrd /initrd.img
# Windows
# this entry searches Windows on the HDD and boot it up
title Windows\nBoot up Windows if installed
  errorcheck off
  find --set-root --ignore-floppies --ignore-cd  /bootmgr
  chainloader /bootmgr
  find --set-root --ignore-floppies --ignore-cd  /ntldr
  chainloader /ntldr
  find --set-root --ignore-floppies --ignore-cd   /io.sys
  chainloader /io.sys
  errorcheck on

jeudi 7 décembre 2017

Recreation : Logitech "gamer" mouse M-U0007

I found one of these old USB "laser" mice in a thrift store for 1€50 and couldn't resist... the one I use on kid2 still has a rubber ball and is pretty old (though it still works).

Anyways, this mouse species has many more buttons and I wanted to try to customize some of them.  So I found a question with the answer at Ask Ubuntu and tried it... with this:


luc@kid2:~$ head -10 .xbindkeysrc 
# For the benefit of emacs users: -*- shell-script -*-
###########################
# xbindkeys configuration #
###########################
# PageUp / PageDown by pushing the wheel left/right.
"xte 'key Page_Up'"
b:6
"xte 'key Page_Down'"
b:7

mercredi 6 décembre 2017

Puppylinux full install to internal (ATA) hard disk

So as I mentioned already, I'm playing with Puppylinux on a small hard drive in kid2, my oldest (still working) desktop computer.

I'm mentioning here some difficulties I had during the install and especially to make it boot properly.  Please note that I'm a complete newbie at Puppylinux, grub4dos, and even to the grub2 configuration/customization that was required to finalize the boot process and keep both Ubuntu and Puppylinux happy, and not destroying the Windows partitions that I still have in this box too.

  1. Before starting the install, you'll need the installation media.  I chose to install from a USB card reader (I have plenty of SD cards laying around but no USB stick).  The installation media for the version that I chose (7.5) came in the form of a *.iso file.  However, it turns out that these are not CD images (ISO9660) but rather they appear to be image copies of the SD card/USB stick.  The Linux file command told me that these were MBR formatted disks so I just used dd to copy the file to an SD card.
  2. During the install attempt, nothing went wrong really, and I was left with a bootable "frugal" install on my 6GB Maxtor PATA disk.  So far so good.
  3. I then tried to configure the Ubuntu disk loader (GRUB2) to boot Puppylinux from this disk.  This proved to be difficult, if only because of the Puppylinux SFS mechanics (layered filesystems mounted using loop(s)).  After some frustration I decided to re-do the install and make it a "full" install (in which the system is booted from a /boot directory directly, and all files are present in the root filesystem like on other distributions that I know).
  4. During a first attempt at a full install, something went wrong... I didn't immediately understand that I need to have the USB installation media (still in the drive) manually (re-)mounted so that the installer could uncompress and copy everything on the "bare" root filesystem.  During my second attempt, I mounted my SD card under /mnt/cdrom so that I could select "CDROM" in the installer as the source for this.
  5. After that attempt, I was still stumbling on the grub4dos configuration.  Then I understood that the initrd.gz file does contain the code to do the filesystem layering required by a "frugal" installation, and apparently required the SFS files to be present (which I didn't want, as my point was to get rid of these now that I have a "full" install).
  6. So I removed the initrd command from grub4dos menu.lst : that worked.  It gave me a hint at reconfiguring also grub2 on the Ubuntu disk, so that I could use both boot loaders as backups of each other.  That part I still have to complete.  In the meantime, below is my Grub4Dos menu entry to boot Puppylinux from the Maxtor:
title Xenialpup64 7.5 (sde1/boot)
uuid abf7b3c6-246c-4189-82a2-a8d41784ba8d
kernel /boot/vmlinuz root=/dev/sde1 ro

(please note that there is no "initrd" command in the above menu entry definition)

kid2 role, revisited

So on the ACPI side, the BIOS+LXde settings seem to be able to suspend the system, resuming (sometimes) with a double click on the mouse.  However this is not what I'd want from a server that is supposed to do "stuff" at periodic 1-hour intervals, like backuppc or my photo posting script.  So far powernap doesn't seem to do anything for me neither, so I removed it completely.

I tested IDE HDDs that were lying around.  Seems like the 500GB and the 200GB are both dead.  I'm left with an old 6GB to play with.  Mounted it on kid2, checked that there was no data left (moved some to kid2 home).  I also moved data from the Windows partition into the home partition so that I can prepare to attempt a Windows reinstall.

On the 6GB hdd I removed the old partitioning, created a single partition, and installed Puppylinux just to play with it, and have an alternate/backup boot method for kid2.  That distro is really damn small!  But quite functional as a intetnet/word processing box.  I might try that on my old eCafé that's sitting in my drawer awaiting for better days.  The installation media could be a single SD card, 512Mb would be enough...  I think the eCafé can boot from USB so that would seem like a piece of cake.

I like the internal hardware on kid2, and the way it allows to add many more 3.5" disks in the future if I wanted to.  The base hardware (packard bell) really has only one free spot left for that.  I'm starting to think about swapping roles for these two boxes.  Both could accept the USB3 PCIe card that I've ordered...  both hardware seem to be reliable still... I wonder how the slower CPU would work in the context of rsync/gzip/network backups; we'll see when I complete the backuppc setup by adding a few more clients on kid2's backup hosts list.

vendredi 1 décembre 2017

kid2 recovery, continued

Upgraded to 17.10 (with intermediate steps).
Kind of works now, except the DHCP that runs at boot doesn't seem to get the DNS name resolution right.  Using
sudo dhclient -v
seems to make it work though.  I also learned not to put eth1 in /etc/network/interfaces when using network-manager to handle it.  I might remove network-manager altogether :)

I did install backuppc and powernap (willing to try this combo for possible power savings).  The install also added apache2 and I changed backuppc's htpasswd after installation.  The postfix email config is probably very wrong.

mercredi 29 novembre 2017

kid2 recovery

Once I hooked up everything from screen to mouse, kid2 wouldn't boot.  I juggled the RAM around a little bit, now both on same-color sockets, then it would still not boot but at least display a BIOS error message on the screen.  So I loaded the "Safe defaults" then "Optimized defaults" (whatever these are) and it worked.
The Windows partition still doesn't work (reboots when I try) but Ubuntu Wily (15.10) is still there.   I had to look a little bit around in order to find a mirror that still had Wily files but then "apt update" worked (except for some ubuntu-security stuff) and I launched "apt dist-upgrade".  That's going to keep it busy for a while.

The only thing that doesn't work right now (in addition to Windows) is the mouse (still a ps/2 ball mouse).  I need to recycle that thing.


  • /dev/sda seems to be a single-partition linux disk (mounted at /home) of size 300G, by Seagate
  • /dev/sdb is smaller (160 Gb Hitachi) and partitioned as follows:
    • sdb1 12 Gb= Windows recovery
    • sdb2 80 Gb= Windows (broken), may still contain some data
    • sdb3 4,2 Gb= swap
    • sdb4 38 Gb= Linux (root)
  • CPU is a E2140 @ 1.60 Ghz
  • There are 3 free PCI slots
  • There's an (unused) IDE interface on the motherboard
  • There are USB slots available, looks like 4 USB1 controllers and 1 USB2 controller
  • The Realtek Ethernet controller claims Gigabit ethernet compatibility
  • There is no built-in graphics card, but an AMD/ATI Radeon is on the PCIe slot

mardi 28 novembre 2017

Backuppc archives timeout problem + new archive HDD medium

So I took delivery of my new Maxtor M3 2Tb USB 3 drive yesterday.  Still the server only has USB 2 ports so I know this is going to be still slow until I can upgrade that.
However I hooked the M3 and it powered up and connected without problems.
fdisk showed a single partition (1.8Gb) formatted as HPFS/NTFS.  I moved the contents to /home/luc/Texts/Manuals or somewhere like that.  It contained some backup software and the documentation apparently.  Then I reformatted using ext4, and changed the partition type to Linux 0x83 (and removed the bootable flag).
Afterthat I used the UUID (from blkid) so add a new line to /etc/auto.removable and it just worked under autofs.  created the backuppc directory and will use that in a new backuppc archive host.
But before that I'd like to fill the iomega HDD with some recent backup archives so that I can move that out of site.  I noticed my previous attempts all failed after 20 hours exactly so after googling a bit I found out about the Client Timeout setting in Backuppc.  Changed this to 48 hours and started a new archive again, from kid3 backup.
To be continued...

kid2 status

I didn't try booting but I opened the (unbranded) tower to check the motherboard model (GA-945PL-DS3) and installed memory (1Gb+1Gb+512Mb of DDR2 PC2-6400). According to the Gigabyte site, this board only supports a maximum of 2GB anyway, so I removed the 512MB DIMM.  I'll need to get a power cable and hook up a screen, power cord and mouse to try and reboot it.  I'll probably need to reset my overclocking attempts in the BIOS too.  On the other hand, I now have switches and Cat5 cables around so I will not need to play with the cheap Wifi dongle that I found back in one of the USB ports.  I didn't bother to check wich hard drives are installed (2xSATA drives are connected).  As there is an IDE interface too, I could also install some more disks in there if needed (doubtful).
I really should try and find a SBC capable of replacing base with a much smaller environmental footprint... but then I'd still need power adapters to feed the drives that I need.  So probably a dead end route.  I could also re-test the power management (powering off via cron when not in use and waking up hourly).  That was kind of fun, but unfortunately it sometimes froze and I didn't troubleshoot that to see if I could easily fix it or not.

lundi 27 novembre 2017

Ubuntu 17.10

Last month I upgraded kid3 & run12l.  No issue, even got the latest Darktable etc so I didn't need to add any ppa so far.  Yesterday I also upgraded base so all OK for now.  I took a look inside (this is a Packard Bell B2501 BE).  Still spare room for disks (both SATA-several connectors free and one free IDE connector) and cards (no extension cards on the motherboard, using the on-board audio and video circuits).  There are 2 PCI slots and at least two PCIe slots (1x and longer).  There's also only 1 DIMM slot used and thus the second one is free.  Looks like this required DIMM DDR2 800.  Iirc the CPU is an E2200.  There's an unused HDD in that box, with only Linux partitions (/ swap /home probably) and it's about 200GB.  It still works (I connected it temporarily for testing) and seems to have data from 2014.  I think /home is empty though so nothing to save from there.  Could probably be used as recovery boot in case of failure of the main SATA HDD.

I ordered a PCIe 4-port USB3 card to install in this box in order to speed up the archives so I can start rolling out with my off-site backups.  I also bought a Maxtor M3 2Tb USB3 external hard disk (at Black Friday prices, 63€) but I still have to pick that up.

I might buy a used monitor screen to replace the one used by kid3 so I can move that one to base in order to troubleshoot a boot failure that I had when the IDE drive was connected (seen as /dev/sda).

dimanche 19 novembre 2017

Automounting removable USB disks for use as backuppc archive volumes

I want to be able to plug in a few (at least 2) removable HDD drives and use them to store my latest backups, so I can easily move these HDD drives offsite for additional protection against theft, fire or other catastrophic events.

Required software

 sudo apt install autofs

/etc/auto.master 

#
# Maps for external HDD etc 
/var/backups/backuppcv /etc/auto.removable --timeout=2 

/etc/auto.removable

Currently I own two of these drives (Iomega is 1TB and Targa is 512MB).  Both are quite old but I plan to add a 2TB (or a couple of them) in the near future.  I did check the blkid from each of these drives then added these lines to my new auto.removable config file:
#
# Maps for external HDD etc
#
iomega    -fstype=ext2    :UUID=e66dd072-7a29-4021-8cea-6757bf3b566a
targa    -fstype=ext3    :UUID=bd524c4e-2c59-4685-9458-74b21a9c33cd


Activation


sudo service autofs restart

Permissions

mkdir /var/backups/backuppcv/iomega/backuppc
chown backuppc:backuppc /var/backups/backuppcv/iomega/backuppc

Usage with backuppc

sudo cat arc_iomega.pl
$Conf{XferMethod} = 'archive';
$Conf{ArchiveDest} = '/var/backups/backuppcv/iomega/backuppc';