The following summary is a braindump of many, many pages from the internet and the tipps and tricks they descriped.

IO Benchmarsks: bonnie++ - IOMeter

RedHat 6 Deployment notes for high performance and Performance Tuning Guide

Get to know your hardware first!

The Hardinfo tool might help.

IO Check out your system. Does it wait for data? # top if the %wa column is that high - you have a problem

ext3/ext4 filesystem disables some ext3 unused features and reduce the amount of journaling data noatime,nodiratime,noacl,data=writeback,commit=15

check this our on a running system without reboot…later you can add the mountoptions to /etc/fstab # mount -o remount,noatime,nodiratime,noacl,commit=15 / # mount -o remount,noatime,nodiratime,noacl,commit=15 /home

be carefully with the journal option. It can only be changed during startup. Test it on a non-productive system first!

another way is to mount the /tmp directory as the TMPFS filesystem. tmpfs /tmp tmpfs defaults,size=256M,noatime,noacl,nodiratime 0 0

ext4 has additional options # mount -o remount,barrier=0 /home

the last bit of performance can be achieved by disabling the journaling, but be aware that the checkdisk may take forever… # mkfs -V -t ext4 /dev/sdb1 # tune2fs -c0 -i0 -m0 /dev/sdb1 # tune2fs -o journal_data_writeback /dev/sdb1 # tune2fs -O ^has_journal /dev/sdb1 /dev/sdb1 /mountpoint ext4 defaults,data=writeback,noatime,nodiratime,noacl,barrier=0 0 0

you can check the active features # tune2fs -l /dev/sdb1

On a busy server the directory index needs to be clean up from time to time. Check if the Options is really active. The filesystem needs to be OFFLINE! # tune2fs -O dir_index /dev/sda5 # umount /dev/sda5 # e2fsck -D -f /dev/sda5

Free up more space to use

reduces the root-reserved space on this volume from 5% to 2%. tune2fs -m2 /dev/sda1

50% of your disk space used but only 2% of the possible inodes?

Then you should optimize your formating…. mkfs -t ext4 /dev/vg00/lvol0 -i 65536

This reduces the inodes about 8 times. The default settings give 1 inode every 8192 bytes. Now only every 64K.

ext3 basics and tuning - ext4 on SSD - linux on flash

IO scheduler

redhat manual

Using fast SSD or external SAN Storage - free up CPU time by minimizing the I/O optimizsation during the scheduling. # echo “noop” > /sys/block/<device>/queue/scheduler return to default # echo “cfq” > /sys/block/<device>/queue/scheduler

depending on the workload - deadline might boost the throughput too - as recommended by Oracle.

IDE/Sata parameters Use hdparm to optimize the disk usage # hdparm -i /dev/sda Parameter auslesen # hdparm -I /dev/sda more infos # hdparm -Tt /dev/sda Geschwindigkeit testen

disable the power- and/or accustic management (if supported) # hdparm -B 255 /dev/sda setting Advanced Power Management level to disabled APM_level = off # hdparm -M 254 /dev/sda setting acoustic management to 254

Configuring Disk Read-Ahead Set read-ahead cache to 512KB /sbin/blockdev –setra 1024 /dev/cciss/c0d0 /sbin/blockdev –setra 1024 /dev/cciss/c0d1

NIC &TCP/IP

Check the nic settings first - with ethtool # ethtool eth0 If it 1000MB/s and Fullduplex … the go on and check the NIC offload functions. Go on with the driver in use # ethtool -i eth0 google the driver in use for more tuning settings or # modinfo <drivername> an INTEL e1000 drivers can tweak the descriptions values from the default 256 to the present max. value 4096. # ethtool -G rx 4096 tx 4096 # ethtool -g eth0

Now continue with the offload engine # ethtool –show-offload eth0 Change the settings –K –offload Changes the offload parameters of the specified ethernet device. rx on|off Specifies whether RX checksumming should be enabled. tx on|off Specifies whether TX checksumming should be enabled. sg on|off Specifies whether scatter-gather should be enabled. tso on|off Specifies whether TCP segmentation offload should be enabled. ufo on|off Specifies whether UDP fragmentation offload should be enabled gso on|off Specifies whether generic segmentation offload should be enabled

Another NIC option is the transmit queue length. # ifconfig eth0 | grep “txqueuelen” ifconfig eth0 txqueuelen 2000

TCP Segmentation offload

The TCP/IP stack can be modified for bigger buffers and more possible window ranges. # sysctl net.core.rmem_max # sysctl net.core.wmem_max # sysctl -w net.core.rmem_max=16777216 # sysctl -w net.core.wmem_max=16777216 On SAP Systems # sysctl -w net.ipv4.ip_local_port_range=“1024 65000” # sysctl -w net.core.rmem_default=262144 # sysctl -w net.core.wmem_default=262144 # sysctl -w net.core.rmem_max=262144 # sysctl -w net.core.wmem_max=262144 # sysctl net.ipv4.tcp_rmem # sysctl net.ipv4.tcp_wmem # sysctl -w net.ipv4.tcp_rmem=“4096 87380 16777216” # sysctl -w net.ipv4.tcp_wmem=“4096 65536 16777216”

Depending on the kernel in use, be sure that the following auto-tuning settins are active. # sysctl net.ipv4.tcp_window_scaling # sysctl net.ipv4.tcp_timestamps # sysctl net.ipv4.tcp_sack

If possible from the hardware slots and BIOS DO NOT SHARE IRQ amoung disk- and network controller(s).

Increase - if possible - the MTU ~9000 (aka. Jumbo Frames) # ping -s 8400 -M do <ip address> c:> ping -f -l 8000 <ip address>

Some CISCO commands - depending on the model config

  mtu 9216

enable

  system mtu 8000

and verify show port jumbo show system mtu

syslog Tell syslog not to sync the disc after every entry. Add an - to the configuration. *.info;mail.none;authpriv.none;cron.none -/var/log/messages If IO is really an issue - you can try disable some rarly used log files - such as debug, cron or maillog

Get rid of the – MARK – entries in syslog use -m 0 option

Need more IO? Try changing to RAID-1 (e.g. from RAID-5) or even RAID-10 or get a SAN-Storage…

Wikipedia ext3 - hdparm - syslog - Lesswatts DISK - iozone.org Filesystem benchmark

memory management tune some kernel parameters Using sysctl.conf to modify some kernel memory options - try it only after studying the online documentation and measuring the results! vm.dirty_writeback_centisecs=1500 vm.swappiness=1 vm.vfs_cache_pressure=50

oracle suggests the following values # less filesystemcache [oracle disables it on some files anyway] in favors of loaded programs vm.swappiness=10 # start flushing out dirty memory pages at 3% of the system memory using pdflush vm.dirty_background_ratio=3 # start flusing out dirty filesystem cache pages at 15% of the system memory vm.dirty_ratio=15 # after 5sec a dirty page has to be written to disk vm.dirty_expire_centisecs=500 # wake up pdflush every second to check if there are any dirty pages to flush vm.dirty_writeback_centisecs=100

to make the change permanent - add them to /etc/sysctl.conf and activate them # sysctl -p

vm-tuning - pdflush theory - Kernel vm.txt

disable unused services and processes The classic phrase, disable all unneeded services - I think you have heard it already. $ who -r $ ps ax

Check out the startup procedures (on Fedora/Centos/RedHat Enterprise) # chkconfig –list

Disable unused services… # chkconfig SERVICENAME off # service SERVICENAME stop

Most of the distributions start several login consoles during /etc/inittab. More than 2 should not really being needed. c1:1235:respawn:/sbin/agetty 38400 tty1 linux c2:1235:respawn:/sbin/agetty 38400 tty2 linux #c3:1235:respawn:/sbin/agetty 38400 tty3 linux #c4:1235:respawn:/sbin/agetty 38400 tty4 linux #c5:1235:respawn:/sbin/agetty 38400 tty5 linux #c6:12345:respawn:/sbin/agetty 38400 tty6 linux

let init reload the new inittab # telinit q if the services do not stop by itself - kill them…

cron jobs Especially if your computer runs 24×7 - keep an eye on the cronjobs [monthly, weekly, daily and the rest]. Time them not to run on the same time as the backup.

Try to use a tickless - kernel. Besides much better powersaving, the kernel doesnt need to poll its internal todolist and with the much improved schedules in 2.6 the whole system should be more responsive.

do your own compilations compiler options I you compile your own binaries - use the compiler switches to optimize the binaries onto your own instrastructure. $ cat /proc/cpuinfo CFLAGS=“-march=prescott -O3 -pipe -fomit-frame-pointer”

in more recent gcc versions there is a generic way without the need to know the exact type of cpu CFLAGS=“-march=native -O3 -pipe -fomit-frame-pointer”

in doubt - check the CPUs capability - either using the kernel or gcc itself # cat /proc/cpuinfo | grep flags flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt npt lbrv svm_lock nrip_save # gcc -c -Q -march=native –help=target The following options are target specific:

1. m128bit-long-double [disabled]
2. m32 [disabled]
3. m3dnow [disabled]
4. m3dnowa [disabled]
5. m64 [enabled]
6. m80387 [enabled]
7. m96bit-long-double [enabled]

…

GNU Compiler Collection Summary - more safe CFLAGS

Usage samples apache / php

Shared vs. static build - Tutorial

do not forget to check if the binaries can be stripped # strip /opt/rrdtools/bin/*

enable only used modules

Doing your own compilation you get the chance to enable only the modules needed and disable the rest.

This will reduce the size of the binary and let you build a lean system $ ./configure –enable-ssl –disable-userdir –disable-imagemap …

consult the online help for the possible configurations or do … $ ./configure –help

kernel The most useful hit ever - build your own kernel. Try to include the necessary drivers als buildin - not as module. strip all not used parts, such as ATM, ISDN… # cat /proc/version # cat /proc/cpuinfo # cat /proc/modules

To start with - look at the file /boot/config-$(uname -r) of the installed distribution. # make localmodconfig on certain systems you need to update the initrd or # make localyesconfig to add all modules static in the kernel

produces a new config with only the present active modules configured - this gives you a very minimal kernel! # make -j 4 bzImage modules # make modules_install headers_install install

If you need to crosscompile a kernel… [target] # lsmod > module_ziel [compile] # make LSMOD=module_ziel localmodconfig [compile] # make bzImage copy the whole /usr/src/linux tree to the target server [target] # make modules_install install

the kernel optimization knowhow is based on an article from the german cT magazin 05/2011 on page 182

applications apache get an overview of the apache at hand # ./apachectl -v # apachectl -M dump all modules # apachectl -S dump all virtual hosts

And do not forget - as always after modifying the httpd.conf or any included files - run a synatex check and reload the service graceful. # apachectl -t # apachectl -k graceful

Multi-Processing Modules (MPMs)

Using linux the preferred and traditional MPM is called prefork. <IfModule mpm_prefork_module>_ StartServers 2 MinSpareServers 1 MaxSpareServers 1 MaxClients 10 MaxRequestsPerChild 0 </IfModule>

The default settings start at least 5 processes with another 5 spare processes. On a system with only few users … way to much.

Newer version work more efficient with the worker MPM ./configure –with-mpm=worker <IfModule mpm_worker_module> StartServers 2 MaxClients 150 MinSpareThreads 10 MaxSpareThreads 50 ThreadsPerChild 25 MaxRequestsPerChild 0 </IfModule>

Reduce IO and network traffic by preventing, caching and compression

disable .htaccess usage AllowOverride None

on a website there are always some static files such as icons, images, styles and scripts. Why not cache them on the browsers! This will reduce the load on the server and allows faster display. <Directory /some/path/to/scripts_images_styles> ExpiresActive On ExpiresByType image/x-icon “access plus 20 minutes” ExpiresByType image/gif “access plus 20 minutes” ExpiresByType image/jpg “access plus 20 minutes” ExpiresByType image/jpeg “access plus 20 minutes” ExpiresByType image/png “access plus 20 minutes” ExpiresByType text/css “access plus 10 minutes” ExpiresByType text/javascript “access plus 20 minutes” ExpiresByType text/x-javascript “access plus 20 minutes” ExpiresByType application/javascript “access plus 20 minutes” ExpiresByType application/x-javascript “access plus 20 minutes” ExpiresDefault “now” </Directory>

reduce the amount of traffic to transfer by compressing it before sending AddOutputFilterByType DEFLATE text/plain AddOutputFilterByType DEFLATE text/html AddOutputFilterByType DEFLATE text/xml AddOutputFilterByType DEFLATE text/css AddOutputFilterByType DEFLATE image/x-icon AddOutputFilterByType DEFLATE application/xml AddOutputFilterByType DEFLATE application/xhtml+xml AddOutputFilterByType DEFLATE application/rss+xml AddOutputFilterByType DEFLATE text/javascript AddOutputFilterByType DEFLATE text/x-javascript AddOutputFilterByType DEFLATE application/javascript AddOutputFilterByType DEFLATE application/x-javascript

A simple online web page compression / deflate / gzip test tool - How To Optimize Your Site With GZIP Compression

Logging

Apache logs most activities. Not all are needed…. Check out the options # LogLevel: Control the number of messages logged to the error_log. # Possible values include: debug, info, notice, warn, error, crit, alert, emerg. LogLevel warn LogFormat “%h %l %u %t ”%r“ %>s %b” common CustomLog “logs/access_log” common

to fit your requirements.

disable the logging completely use CustomLog /some/path combined env=DOES_NOT_EXIST

httpd.apache.org

mysql

Mysql can be quite a beast - especially if you host several Typo3 CMS on a server with a DB volume of over 3GByte. A look at its numbers is worth a while. # mysql –version

Before you start - make sure you have a working backup - just in case $ mysqldump –user=<name> –password=<pwd> –all-databases –opt –result-file=all.sql or $ mysqldump –user=<name> –password=<pwd> <db> –opt –result-file=<db>.sql

Start with this presentation (german)

Use the tuning-primer.sh (website) script to optimize some key options. (sample)

Next would be to disable the binary log. It is used for cluster replication and point in time restores. If you use neither nor - disable it #log-bin

A reset might be handy before disabling to reduce the logfile-size reset master;

Another area are the DB-Engines. The newer InnoDB module adds a lot of sql functionality to mysql for the price of performance.

If you do not need the additional sql - disable it and stick to the traditional, fast Isamchk. disable all #innodb_… entries in my.cnf and add skip-innodb

Be aware that InnoDB tables have to be converted! alter table tblName ENGINE=MYISAM;

innoDB tutorial

Is mysql still hogging your CPU? Try cleaning up the db content. Sometime there are cache tables, outdated indexes or fragmentated databases to clean up.

First get a picture of the tables available. show table status like index_fulltext;

If you have a lot of tables - you might use the showtablesize.sh to automatically print out all tables in the database given. [show table status reference]

Sometimes, people forget the maintenance tasks - so the sysadmin has to do the cleanup. e.g. for index_ and cache_ tables in typo3, use the truncate function to clean them [truncate reference] truncate table index_fulltext;

After all the cleanup optimize the tables to regain full performance.

BE AWARE that during optimizing the tables are LOCKED by mysql. optimize table index_fulltext;

[optimize table reference]

additional performance can be achieved by optimizing the SQL code - or to be more precise use additional attribute indexes to tune the SQL internal operation.

e.g. for Cacti CREATE INDEX `data_template_data_id` ON `data_input_data` (`data_template_data_id`); … CREATE INDEX `host_id_rrd_next_step` ON poller_item (`host_id`,`rrd_next_step`);

MySql 5.0 Reference - mysqltuner.com alternative checkscript - optimize Tables with cronjob - some handy sql statements - mysql basic troubleshooting

php

configuration

be sure to enable persistent mysql connections… mysql.allow_persistent = On mysql.max_persistent = -1 mysql.max_links = -1 mysql.connect_timeout = 60 mysql.trace_mode = Off

misc… startup time If not needed, disable the kernel chooser prompt during LILO/GRUB and put the kernel in Quiet-Mode. #prompt append = “ quiet”

Boot Time # vi lilo.conf compact

Also try to “compact” BIOS read calls, as there are the slowest in the world….

disable some default ext3 check during startup - but do not forget to do them manually during a service window! # tune2fs -c 0 -i 0 /dev/vg00/lvol1

Tracing the most powerconsuming tasks with powertop.

- Kernel Options

- X11 Device Option: Option “NoDRI” reduces the RADEON Interrups

- USB Autosuspend : usbcore.autosuspend=1

- CPU Governor ondemand for both cpus..

- setterm in rc.M