Continuing
from Linux Home
Server 2008 [md080226].html
Which Linux? Best Linux. Hardware
RAID. Disk Performance. Linux
home server. Highpoint 2300 RAID. Linux Media
Server PC
090820
Noticed remotely that the root file
system was full, some maintenance and fsck wouldn’t
fix; did remote reboot and lost access
Got home, did more fsck’s, boot disk wrecked; device tree damaged so no
networking.
Tried repairs with Fedora8 – doesn’t see drives; tried
Fedora11, and that reset my grub menu so no more booting – did offer to remove
my invalid partitions too which I didn’t do!
Server down,
and I’m short on time.
0905
Added quad core Q9550. Very nice
and painless.
080504
Unexplained crash, rebooted to HPT RAID screen, and hung –
display shifted. Hard poweroff/restart.
All OK
No logging to indicated cause. Was in the middle of a backup – so HPT tripped up ?
Add R-U-ON
080226
·
Most
file copies complete. Tried to using Seagate & Western
Digital utilities to find Acoustic/Power saving options. Seatools had no facilities except disk check, Western
Digital diag couldn’t find
drive (when connected to motherboard), so abandoned.
·
Installed Marvell network controller. Can’t ping same segment –
work on later
·
Adding
patches to Fedora (except Kernel). Check to see if concurrency/threading improves. Discovered disk tuning!
·
Tuning:
I think my big RAM is creating bad behaviour in the Linux caching defaults.
Changes:
o Found http://www.3ware.com/kb/article.aspx?id=11050
& http://hep.kbfi.ee/index.php/IT/KernelTuning
o #1 /proc/sys/vm/swappiness (default 60, set to 20): encourages more app
caching instead of disk buffer (more frequent smaller flushing)
o #2
/proc/sys/vm/dirty_background_ratio (default 5%, set
to 20%): increase max dirty buffers, so flushing is less frequent (stop I/O
writes overloading system).
o /proc/sys/vm/dirty_ratio (default 10, set to 60): reduced write
throttling
o add all settings to: /etc/rc.d/rc.local:
o …
much nicer now, VM’s very
happy too, mostly.
§ #disk tuning &
performance options md080225
§ echo 256 > /sys/block/sda/queue/nr_requests
§ blockdev --setra
32768 /dev/sda
§ echo 20 > /proc/sys/vm/swappiness
§ echo 20 > /proc/sys/vm/dirty_background_ratio
§ echo 60 > /proc/sys/vm/dirty_ratio
§ echo deadline >
/sys/block/sda/queue/scheduler
·
Look at xfsdump and xfsrestore.
·
Add the DNLA server. http://www.twonkyvision.com/ (which
works with Xbox 360 too)
·
Future options: Autumn 08+. Move system to Mandriva - use 2.5” SATA and bury it in a spare space
(retain Fedora boot as backup, maybe test kernel upgrade), work through device
drivers. Then test disk performance/thread lucidity – with Fedora there are now
only occasional hiccups but still there. Also a quad
core is an option.
·
**
keep an eye on Seagate #2 it does have sticking head (jolly old eBay), but always recovers – does rarely stall OS under
high load, but recovers in 500ms.
·
To
mitigate the huge build time this server must now run for 4 years. No changes till
March 2012. Currently 700GB used, 700GB free. Except for the
quad CPU of course, but not yet.
·
DON’T upgrade Kernel, will likely disable VMware
or worse HPT RAID controller.
-
Replace: Sun LX50 running Sun Java
Desktop Linux (based on Suse9) with HPT1820a + RAID5 3 x 160GB SATA + Single
300GB SCSI.
Over
3 years, one loss of drive, which was replaced with a 250GB inflight. 160GB after testing seemed to be fine.
Gnome desktop failed on year 3 (somehow destroyed it's
config), reverted to basic Xwindow
manager.
New: Abit
Fatal1ty I90 HD + 3 x 750GB SATA. Tried cheap XFX SATA RAID3 controller
(www.scan.co.uk), but can't handle Core2 Duo CPU >
1.6Ghz in this board.
Drive
#1 An enterprise Seagate 750es #2 Seagate 750 seems to elicit weird noises (Ebay purchase), not sure about the longevity. #3 Western
Digital run up large amount of early errors (brand new), but has stabilised
(under WHS). WD not compatible with XFX SATA RAID3
controller. My first experimental case didn’t
favour the drives which seem to transmit vibration to each other (the drives
would stall/hesitate) – new media case has damping for all drive mounts.
Has onboard Realtek Gigabit, and added a Marvell
Gigabit PCI-e card, as indications was this would have
decent support.
Abit
Fatal1ty I90 HD with Radeon Xpress
1250 Video. HDMI upset all distributions (initial testing), but using
VGA anyway now, and new ECS motherboard less exotic.
Case:
Antek NSK2400 (http://www.silentpcreview.com/article591-page1.html)
though replaced PSU with 450W unit during early isolation testing with the XFS
card’s weird behaviour. Could switch back to the supplied
380W as I have configured HPT 2300 for staggered drive starts. Have rotated both main right fans so they blow inwards. Many
vent holes covered to control push airflow out via drive bay. 3rd WD drive sits above PSU on high rubber feet –
whitetack securing to rear of case. The fans in this config provide support for the PSU/3rd Drive
(separating hole at minimum closure). The low speed PSU fan didn’t
work so well with main fains pulling (though I used
one only initially).
New#2:
ECS G33T-M2. Onboard Realtek & Intel
GMA 300 video
Board good. Of note: supports PCI IO remapping
to high memory so will I get all 6GB ? Reports MemTotal: 6124000 kB, so only losing 164MB. Overclocking
facilities are zero, but that stops me experimenting, and it does have a harsh
operating enviro (the Loft – summer is a killer!)
HPT2300
doesn’t appear to support INT13 boot, which is CRAP.
Means I have to mess about with driver add processes – which with Linux in play
will be a nightmare (something the HPT1820a handled so well). Maybe a PCIe constraint ?
Hours
to install, and personalised to requirements (run VMware
& share some drives basically):
WHS:
10hrs (next time Win2k3 server !)
Linux:
100hrs
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Need
more space (from 300GB from the array above).
First tried WHS (Windows Home Server), but indications are lack of
reliability, risk of data loss at high loads (it's
running multiple VM's and associated installs).
Whilst dynamic adds of storage is good, it can't
protect root/boot partition, and rebuild time would be horrendous for big
drives. The lazy write/balancing has the drives doing stuff constantly.
Initial
Software RAID test
·
3GB
RAID 0 SWAP OK
·
8GB
RAID 1 / OK
·
1400GB
RAID 5 /share OK
Must have built in support
for an available network card. Only Fedora complies, and in fact supports the onboard and
the addon Marvell.
Must support XFS natively (I trust it for big data, and it's roots. Supports fast deletes).
Must run VMware server
without failed compilation or other grief.
OpenSuse Linux x64
10.3: Can't produce bootable software RAID config; Can boot with aid of CD; No onboard or addon Ethernet. Does
support video card. DEAD
Ubuntu x64 7.1: Can't
boot video mode or fallback text mode. DEAD. I like the lightweight CD, then install +
network package add concept. Allows very fast testing.
Ubuntu x86 7.1: OK, but 32bit VM can’t run
64bit guest (Vista32 on a 64bit CPU can). Wanted x64.
DEAD
Fedora 8: Does
support addon Ethernet. Does
support XFS. Text mode only (Abit I90). Can’t compile VMware
Server 1.0.4 (and VMS2.0 beta(alpha!) has broken
console). Found a patch, so VMware 1.0.4 is running
OK.
CentOS 5: Can't
produce bootable software RAID config. Does NOT support XFS. Does support video
card. No onboard or addon Ethernet. DEAD
Mandriva x64: No onboard
or addon Ethernet. Best setup GUI. Inherited RAID partitions from Fedora install,
even tried to upgrade Fedora (but failed).
Supports XFS. USB memory automount/popup
is good. OS felt very fast and lucid – and RAID disk IO was 2x
that of Fedora. Shame about network cards. DEAD
Final HPT RAID 2300 layout:
/dev/sda1 194442 17258 167145 10%
/boot
/dev/sda2 10154020 5880600 3749300 62%
/
/dev/sda3 6289436 swap
/dev/sda5 1447887616 744635444 703252172 52%
/share
Get
HPT2300 PCIe card ? In Progress. Arrives 080208. OK, but not driverless card so inferior user experience to the
1820a.
Ubuntu x64 with addon GFX card ? Nope.
Revert
to previous Fedora (with HPT2300 above), and test VMware
& networking ? OK. This option used.
Added
2 x 2GB Ebay DDR2-800 Dimms
today – so motherboard is reporting 6GB. Cool. Pushed voltage to 2V (max for ECS board) as highload RAM disk testing (/dev/shm)
gave spontaneous reboots at stock. Rock solid now.
With
Mandriva, a USB 10/100 card worked (dynamically) but
it had limited tfr speed ~600KB/s.
Mandriva got annoying, I went down the Kernel recompile to get the addon Marvell Ethernet card working – I wasn’t supposed to
do that, which more than likely led to the next item. Realtek
never worked. Marvell had spectacularly better install process, but eventually
prompted a kernel recompile, but that would never boot. So
all shite, really.
!! BROKE the motherboard. Was trying Ubuntu
x64 next, as I was inserting new graphics card, I dropped it (not from a great
height), and it chipped two capacitors, and it will boot no more (locks after
CPU speed display). I of course blame Linux, since #1 I wouldn’t have been
impatient and angry with all the combinations I had to try and #2 shouldn’t
have needed another card anyway !! £60
bye bye. I could
RMA it, but that would make me an immoral ass.
New
ECS motherboard arrives tomorrow (080208), Ebuyer
typically exceptional with delivery commitments (unlike scan.co.uk). And they did. The floppy connector really is required ->
so I could upgrade the HPT 2300 firmware. Unbelievable.
Fedora
Yum install:
xinetd
vnc-server
gparted
sysstat (for iostat)
gnome-applet-sensors (cpu
speed & temp)
gnome-schedule (gui
cron)
samba-swat
pbzip2
p7zip
Service autostart
Services à enable Samba & NFS
Tools
Handy smart info (though raid
browser tool is good too)
Show: smartctl
--all --device=hpt/1/[1-3] /dev/sda
Test: smartctl
--t short --device=hpt/1/[1-3] /dev/sda
Setup
touch /.autorelabel
Samba SeLinux
setsebool -P nfs_export_all_ro=1
chcon -R -t samba_share_t
*
Disable
SELinux – I tried permissive but it
dumps too much to the logs – renders /var/log/messages
useless.
OS should anticipate
defaults. SELinux Disabled.
Ubuntu Ref
apt-get