Linux Home Server 2009

Continuing from Linux Home Server 2008 [md080226].html

Keywords

Which Linux? Best Linux. Hardware RAID. Disk Performance. Linux home server. Highpoint 2300 RAID. Linux Media Server PC

Log

090820

            Noticed remotely that the root file system was full, some maintenance and fsck wouldn’t fix; did remote reboot and lost access

            Got home, did more fsck’s, boot disk wrecked; device tree damaged so no networking.

            Tried repairs with Fedora8 – doesn’t see drives; tried Fedora11, and that reset my grub menu so no more booting – did offer to remove my invalid partitions too which I didn’t do!

            Server down, and I’m short on time.

0905

Added quad core Q9550. Very nice and painless.
080504

Unexplained crash, rebooted to HPT RAID screen, and hung – display shifted. Hard poweroff/restart. All OK
No logging to indicated cause. Was in the middle of a backup – so HPT tripped up ?

Add R-U-ON

080226

·         Most file copies complete. Tried to using Seagate & Western Digital utilities to find Acoustic/Power saving options. Seatools had no facilities except disk check, Western Digital diag couldn’t find drive (when connected to motherboard), so abandoned.

·         Installed Marvell network controller. Can’t ping same segment – work on later

·         Adding patches to Fedora (except Kernel). Check to see if concurrency/threading improves. Discovered disk tuning!

·         Tuning: I think my big RAM is creating bad behaviour in the Linux caching defaults. Changes:

o   Found http://www.3ware.com/kb/article.aspx?id=11050 & http://hep.kbfi.ee/index.php/IT/KernelTuning

o   #1 /proc/sys/vm/swappiness (default 60, set to 20): encourages more app caching instead of disk buffer (more frequent smaller flushing)

o   #2 /proc/sys/vm/dirty_background_ratio (default 5%, set to 20%): increase max dirty buffers, so flushing is less frequent (stop I/O writes overloading system).

o   /proc/sys/vm/dirty_ratio (default 10, set to 60): reduced write throttling

o   add all settings to: /etc/rc.d/rc.local:

o   much nicer now, VM’s very happy too, mostly.

§  #disk tuning & performance options md080225

§  echo 256 > /sys/block/sda/queue/nr_requests

§  blockdev --setra 32768 /dev/sda

§  echo 20 > /proc/sys/vm/swappiness

§  echo 20 > /proc/sys/vm/dirty_background_ratio

§  echo 60 > /proc/sys/vm/dirty_ratio

§  echo deadline > /sys/block/sda/queue/scheduler

·         Look at xfsdump and xfsrestore.

·         Add the DNLA server. http://www.twonkyvision.com/ (which works with Xbox 360 too)

·         Future options: Autumn 08+. Move system to Mandriva - use 2.5” SATA and bury it in a spare space (retain Fedora boot as backup, maybe test kernel upgrade), work through device drivers. Then test disk performance/thread lucidity – with Fedora there are now only occasional hiccups but still there. Also a quad core is an option.

Reminders

·         ** keep an eye on Seagate #2 it does have sticking head (jolly old eBay), but always recovers – does rarely stall OS under high load, but recovers in 500ms.

·         To mitigate the huge build time this server must now run for 4 years. No changes till March 2012. Currently 700GB used, 700GB free. Except for the quad CPU of course, but not yet.

·         DON’T upgrade Kernel, will likely disable VMware or worse HPT RAID controller.

-

Hardware

Replace: Sun LX50 running Sun Java Desktop Linux (based on Suse9) with HPT1820a + RAID5 3 x 160GB SATA + Single 300GB SCSI.

Over 3 years, one loss of drive, which was replaced with a 250GB inflight. 160GB after testing seemed to be fine.
Gnome desktop failed on year 3 (somehow destroyed it's config), reverted to basic Xwindow manager.

 

New: Abit Fatal1ty I90 HD + 3 x 750GB SATA. Tried cheap XFX SATA RAID3 controller (www.scan.co.uk), but can't handle Core2 Duo CPU > 1.6Ghz in this board.

Drive #1 An enterprise Seagate 750es #2 Seagate 750 seems to elicit weird noises (Ebay purchase), not sure about the longevity. #3 Western Digital run up large amount of early errors (brand new), but has stabilised (under WHS). WD not compatible with XFX SATA RAID3 controller. My first experimental case didn’t favour the drives which seem to transmit vibration to each other (the drives would stall/hesitate) – new media case has damping for all drive mounts.
Has onboard Realtek Gigabit, and added a Marvell Gigabit PCI-e card, as indications was this would have decent support.
Abit Fatal1ty I90 HD with Radeon Xpress 1250 Video. HDMI upset all distributions (initial testing), but using VGA anyway now, and new ECS motherboard less exotic.

Case: Antek NSK2400 (http://www.silentpcreview.com/article591-page1.html) though replaced PSU with 450W unit during early isolation testing with the XFS card’s weird behaviour. Could switch back to the supplied 380W as I have configured HPT 2300 for staggered drive starts. Have rotated both main right fans so they blow inwards. Many vent holes covered to control push airflow out via drive bay. 3rd WD drive sits above PSU on high rubber feet – whitetack securing to rear of case. The fans in this config provide support for the PSU/3rd Drive (separating hole at minimum closure). The low speed PSU fan didn’t work so well with main fains pulling (though I used one only initially).

 

New#2: ECS G33T-M2. Onboard Realtek & Intel GMA 300 video

Board good. Of note: supports PCI IO remapping to high memory so will I get all 6GB ? Reports MemTotal: 6124000 kB, so only losing 164MB. Overclocking facilities are zero, but that stops me experimenting, and it does have a harsh operating enviro (the Loft – summer is a killer!)

HPT2300 doesn’t appear to support INT13 boot, which is CRAP. Means I have to mess about with driver add processes – which with Linux in play will be a nightmare (something the HPT1820a handled so well). Maybe a PCIe constraint ?

 

Hours to install, and personalised to requirements (run VMware & share some drives basically):

WHS: 10hrs (next time Win2k3 server !)

Linux: 100hrs !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Background

Need more space (from 300GB from the array above).
First tried WHS (Windows Home Server), but indications are lack of reliability, risk of data loss at high loads (it's running multiple VM's and associated installs). Whilst dynamic adds of storage is good, it can't protect root/boot partition, and rebuild time would be horrendous for big drives. The lazy write/balancing has the drives doing stuff constantly.

Linux Requirements

Initial Software RAID test

·         3GB RAID 0 SWAP    OK

·         8GB RAID 1 /    OK

·         1400GB RAID 5 /share    OK


Must have built in support for an available network card.
Only Fedora complies, and in fact supports the onboard and the addon Marvell.
Must support XFS natively (I trust it for big data, and it's roots. Supports fast deletes).
Must run VMware server without failed compilation or other grief.

 

OpenSuse Linux x64 10.3: Can't produce bootable software RAID config; Can boot with aid of CD; No onboard or addon Ethernet. Does support video card. DEAD
Ubuntu x64 7.1: Can't boot video mode or fallback text mode. DEAD. I like the lightweight CD, then install + network package add concept. Allows very fast testing.

Ubuntu x86 7.1: OK, but 32bit VM can’t run 64bit guest (Vista32 on a 64bit CPU can). Wanted x64. DEAD
Fedora 8: Does support addon Ethernet. Does support XFS. Text mode only (Abit I90). Can’t compile VMware Server 1.0.4 (and VMS2.0 beta(alpha!) has broken console). Found a patch, so VMware 1.0.4 is running OK.
CentOS 5: Can't produce bootable software RAID configDoes NOT support XFS. Does support video card. No onboard or addon Ethernet. DEAD

Mandriva x64: No onboard or addon Ethernet. Best setup GUI. Inherited RAID partitions from Fedora install, even tried to upgrade Fedora (but failed).
Supports XFS. USB memory automount/popup is good. OS felt very fast and lucid – and RAID disk IO was 2x that of Fedora. Shame about network cards. DEAD

 

Final HPT RAID 2300 layout:

 

/dev/sda1           194442            17258              167145            10% /boot

/dev/sda2           10154020        5880600          3749300          62% /

/dev/sda3           6289436          swap

/dev/sda5           1447887616    744635444      703252172      52% /share

TODO

Get HPT2300 PCIe card ? In Progress. Arrives 080208. OK, but not driverless card so inferior user experience to the 1820a.

Ubuntu x64 with addon GFX card ? Nope.

Revert to previous Fedora (with HPT2300 above), and test VMware & networking ? OK. This option used.

Added 2 x 2GB Ebay DDR2-800 Dimms today – so motherboard is reporting 6GB. Cool. Pushed voltage to 2V (max for ECS board) as highload RAM disk testing (/dev/shm) gave spontaneous reboots at stock. Rock solid now.

With Mandriva, a USB 10/100 card worked (dynamically) but it had limited tfr speed ~600KB/s.

Mandriva got annoying, I went down the Kernel recompile to get the addon Marvell Ethernet card working – I wasn’t supposed to do that, which more than likely led to the next item. Realtek never worked. Marvell had spectacularly better install process, but eventually prompted a kernel recompile, but that would never boot. So all shite, really.

!! BROKE the motherboard. Was trying Ubuntu x64 next, as I was inserting new graphics card, I dropped it (not from a great height), and it chipped two capacitors, and it will boot no more (locks after CPU speed display). I of course blame Linux, since #1 I wouldn’t have been impatient and angry with all the combinations I had to try and #2 shouldn’t have needed another card anyway !! £60 bye bye. I could RMA it, but that would make me an immoral ass.

New ECS motherboard arrives tomorrow (080208), Ebuyer typically exceptional with delivery commitments (unlike scan.co.uk). And they did. The floppy connector really is required -> so I could upgrade the HPT 2300 firmware. Unbelievable.

Observations/Usabiliy

Post Setup

Fedora

Yum install:

 

xinetd

vnc-server
gparted
sysstat (for iostat)

gnome-applet-sensors (cpu speed & temp)

gnome-schedule (gui cron)

samba-swat

pbzip2

p7zip

 

Service autostart

 

Services à enable Samba & NFS

 

Tools

Handy smart info (though raid browser tool is good too)

Show: smartctl --all --device=hpt/1/[1-3] /dev/sda

Test: smartctl --t short --device=hpt/1/[1-3] /dev/sda

 

Setup

 

touch /.autorelabel

 

Samba SeLinux

setsebool -P nfs_export_all_ro=1

chcon -R -t samba_share_t *

 

Disable

 

SELinux – I tried permissive but it dumps too much to the logs – renders /var/log/messages useless.

 OS should anticipate defaults. SELinux Disabled.

 

Ubuntu Ref

apt-get