… continued
at Linux Home
Server 2009 [md090821].html
(hopefully
not March ! – just made it!)
Which
Linux? Best Linux. Hardware RAID. Disk Performance. Linux home server.
Highpoint 2300 RAID. Linux Media Server PC
080504
Unexplained crash, rebooted to HPT RAID screen, and hung –
display shifted. Hard poweroff/restart. All OK
No logging to indicated cause. Was in the middle of a backup – so HPT tripped
up ?
Add R-U-ON
080226
·
Most
file copies complete. Tried to using Seagate & Western Digital utilities to
find Acoustic/Power saving options. Seatools had no facilities except disk
check, Western Digital diag couldn’t find drive (when connected to
motherboard), so abandoned.
·
Installed
Marvell network controller. Can’t ping same segment – work on later
·
Adding
patches to Fedora (except Kernel). Check to see if concurrency/threading
improves. Discovered disk tuning!
·
Tuning:
I think my big RAM is creating bad behaviour in the Linux caching defaults.
Changes:
o Found http://www.3ware.com/kb/article.aspx?id=11050
& http://hep.kbfi.ee/index.php/IT/KernelTuning
o #1 /proc/sys/vm/swappiness
(default 60, set to 20): encourages more app caching instead of disk buffer (more
frequent smaller flushing)
o #2
/proc/sys/vm/dirty_background_ratio (default 5%, set to 20%): increase max
dirty buffers, so flushing is less frequent (stop I/O writes overloading
system).
o /proc/sys/vm/dirty_ratio
(default 10, set to 60): reduced write throttling
o add all settings to: /etc/rc.d/rc.local:
o …
much nicer now, VM’s very happy too, mostly.
§ #disk tuning &
performance options md080225
§ echo 256 >
/sys/block/sda/queue/nr_requests
§ blockdev --setra 32768
/dev/sda
§ echo 20 >
/proc/sys/vm/swappiness
§ echo 20 >
/proc/sys/vm/dirty_background_ratio
§ echo 60 >
/proc/sys/vm/dirty_ratio
§ echo deadline >
/sys/block/sda/queue/scheduler
·
Look at xfsdump and xfsrestore.
·
Add the DNLA server. http://www.twonkyvision.com/ (which
works with Xbox 360 too)
·
Future
options: Autumn 08+. Move system to Mandriva - use 2.5” SATA and bury it in a
spare space (retain Fedora boot as backup, maybe test kernel upgrade), work
through device drivers. Then test disk performance/thread lucidity – with
Fedora there are now only occasional hiccups but still there. Also a quad core
is an option.
·
**
keep an eye on Seagate #2 it does have sticking head (jolly old eBay), but
always recovers – does rarely stall OS under high load, but recovers in 500ms.
·
To
mitigate the huge build time this server must now run for 4 years. No changes till March 2012. Currently
700GB used, 700GB free. Except for the quad CPU of course, but not yet.
·
DON’T
upgrade Kernel, will likely disable VMware or worse HPT RAID controller.
-
Replace:
Sun LX50 running Sun Java Desktop Linux (based on Suse9) with HPT1820a + RAID5
3 x 160GB SATA + Single 300GB SCSI.
Over
3 years, one loss of drive, which was replaced with a 250GB inflight. 160GB
after testing seemed to be fine.
Gnome desktop failed on year 3 (somehow destroyed it's config), reverted to
basic Xwindow manager.
New: Abit Fatal1ty I90 HD + 3
x 750GB SATA. Tried cheap XFX SATA RAID3 controller (www.scan.co.uk), but can't
handle Core2 Duo CPU > 1.6Ghz in this board.
Drive
#1 An enterprise Seagate 750es #2 Seagate 750 seems to elicit weird noises
(Ebay purchase), not sure about the longevity. #3 Western Digital run up large
amount of early errors (brand new), but has stabilised (under WHS). WD not
compatible with XFX SATA RAID3 controller. My first experimental case didn’t
favour the drives which seem to transmit vibration to each other (the drives
would stall/hesitate) – new media case has damping for all drive mounts.
Has onboard Realtek Gigabit, and added a Marvell Gigabit PCI-e card, as
indications was this would have decent support.
Abit Fatal1ty I90 HD with Radeon Xpress 1250 Video. HDMI upset all
distributions (initial testing), but using VGA anyway now, and new ECS
motherboard less exotic.
Case:
Antek NSK2400 (http://www.silentpcreview.com/article591-page1.html)
though replaced PSU with 450W unit during early isolation testing with the XFS
card’s weird behaviour. Could switch back to the supplied 380W as I have
configured HPT 2300 for staggered drive starts. Have rotated both main right
fans so they blow inwards. Many vent holes covered to control push airflow out
via drive bay. 3rd WD drive sits above PSU on high rubber feet –
whitetack securing to rear of case. The fans in this config provide support for
the PSU/3rd Drive (separating hole at minimum closure). The low
speed PSU fan didn’t work so well with main fains pulling (though I used one
only initially).
New#2:
ECS G33T-M2. Onboard Realtek & Intel
GMA 300 video
Board good. Of note: supports
PCI IO remapping to high memory so will I get all 6GB ? Reports MemTotal:
6124000 kB, so only losing 164MB. Overclocking facilities are zero, but that
stops me experimenting, and it does have a harsh operating enviro (the Loft –
summer is a killer!)
HPT2300
doesn’t appear to support INT13 boot, which is CRAP. Means I have to mess about
with driver add processes – which with Linux in play will be a nightmare
(something the HPT1820a handled so well). Maybe a PCIe constraint ?
Hours
to install, and personalised to requirements (run VMware & share some
drives basically):
WHS:
10hrs (next time Win2k3 server !)
Linux:
100hrs !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Need
more space (from 300GB from the array above).
First tried WHS (Windows Home Server), but indications are lack of
reliability, risk of data loss at high loads (it's running multiple VM's and
associated installs). Whilst dynamic adds of storage is good, it can't protect
root/boot partition, and rebuild time would be horrendous for big drives. The
lazy write/balancing has the drives doing stuff constantly.
Initial
Software RAID test
·
3GB
RAID 0 SWAP OK
·
8GB
RAID 1 / OK
·
1400GB
RAID 5 /share OK
Must have
built in support for an available network card. Only Fedora complies, and in
fact supports the onboard and the addon Marvell.
Must support XFS natively (I trust it for big data, and it's roots.
Supports fast deletes).
Must run VMware server without failed compilation or other grief.
OpenSuse Linux x64 10.3: Can't produce bootable software RAID
config; Can boot with aid of CD; No onboard or addon Ethernet.
Does support video card. DEAD
Ubuntu x64 7.1:
Can't boot video mode or fallback text mode. DEAD. I like the lightweight CD, then install +
network package add concept. Allows very fast testing.
Ubuntu x86 7.1: OK, but 32bit VM can’t
run 64bit guest (Vista32 on a 64bit CPU can). Wanted x64. DEAD
Fedora 8: Does
support addon Ethernet. Does support XFS. Text mode
only (Abit I90). Can’t compile VMware Server 1.0.4 (and VMS2.0
beta(alpha!) has broken console). Found a patch, so VMware 1.0.4 is running OK.
CentOS 5: Can't
produce bootable software RAID config. Does NOT support XFS.
Does support video card. No onboard or addon Ethernet.
DEAD
Mandriva
x64: No onboard
or addon Ethernet. Best setup GUI. Inherited RAID partitions from Fedora
install, even tried to upgrade Fedora (but failed).
Supports XFS. USB memory automount/popup is good. OS felt very fast and lucid –
and RAID disk IO was 2x that of Fedora. Shame about network cards. DEAD
Final HPT RAID 2300 layout:
/dev/sda1 194442 17258 167145 10%
/boot
/dev/sda2 10154020 5880600 3749300 62%
/
/dev/sda3 6289436 swap
/dev/sda5 1447887616 744635444 703252172 52%
/share
Get
HPT2300 PCIe card ? In Progress. Arrives 080208. OK, but not driverless card so
inferior user experience to the 1820a.
Ubuntu
x64 with addon GFX card ? Nope.
Revert
to previous Fedora (with HPT2300 above), and test VMware & networking ? OK.
This option used.
Added
2 x 2GB Ebay DDR2-800 Dimms today – so motherboard is reporting 6GB. Cool.
Pushed voltage to 2V (max for ECS board) as highload RAM disk testing
(/dev/shm) gave spontaneous reboots at stock. Rock solid now.
With
Mandriva, a USB 10/100 card worked (dynamically) but it had limited tfr speed
~600KB/s.
Mandriva
got annoying, I went down the Kernel recompile to get the addon Marvell
Ethernet card working – I wasn’t supposed to do that, which more than likely
led to the next item. Realtek never worked. Marvell had spectacularly better
install process, but eventually prompted a kernel recompile, but that would
never boot. So all shite, really.
!!
BROKE the motherboard. Was trying Ubuntu x64 next, as I was inserting new
graphics card, I dropped it (not from a great height), and it chipped two
capacitors, and it will boot no more (locks after CPU speed display). I of
course blame Linux, since #1 I wouldn’t have been impatient and angry with all
the combinations I had to try and #2 shouldn’t have needed another card anyway
!! £60 bye bye. I could RMA it, but
that would make me an immoral ass.
New
ECS motherboard arrives tomorrow (080208), Ebuyer typically exceptional with
delivery commitments (unlike scan.co.uk). And they did. The floppy connector
really is required -> so I could upgrade the HPT 2300 firmware.
Unbelievable.
Fedora
Yum install:
xinetd
vnc-server
gparted
sysstat (for iostat)
gnome-applet-sensors (cpu speed & temp)
gnome-schedule (gui cron)
samba-swat
pbzip2
p7zip
Service autostart
Services à enable Samba & NFS
Tools
Handy smart info (though raid
browser tool is good too)
Show: smartctl --all
--device=hpt/1/[1-3] /dev/sda
Test: smartctl --t short
--device=hpt/1/[1-3] /dev/sda
Setup
touch /.autorelabel
Samba SeLinux
setsebool -P nfs_export_all_ro=1
chcon -R -t samba_share_t *
Disable
SELinux – I tried permissive but it dumps too much to the
logs – renders /var/log/messages useless.
OS should anticipate
defaults. SELinux Disabled.
Ubuntu Ref
apt-get