Here is a good explanation of why you don't need virus protection in Linux
Wednesday, January 9, 2008
Monday, January 7, 2008
Logging
Logging is a way to keep track of events on the server.
A lot of daemons generate their own logs. Look in /var/log.
I have a backup route that uses cron to run rsync once a week. I have it output the changed files and errors to a log file. The cron entry to do this is:
0 1 * * 0 rsync -av --delete /home/documents/ /home/backup/ > rsynclog.$(date +%Y%m%d).txt
lets break that down. The first section (0 1 * * 0) tells cron when to do it. In this case every Sunday at 1:00 am. See the cron entry for more info.
Next is the command (rsync..../backup/) see the NAS backup section for more info on this.
The final section creates the log. The v in -av will output which files are changed added or deleted to the backup set. If you do no more, this will be output to the terminal on the screen and will be gone when you close the terminal. To save it to a file we use the >. > redirects the output to the file specified. In our case this is rsynclog.(todays date).txt. Todays date is generated by $(date) and is formatted by +%Y%m%d. See the man page for the date command for more options.
http://www.unixgeeks.org/security/newbie/unix/cron-1.html
http://www.xaprb.com/blog/2006/06/06/what-does-devnull-21-mean/
http://www.cyberciti.biz/faq/linux-log-files-location-and-how-do-i-view-logs-files/
A lot of daemons generate their own logs. Look in /var/log.
I have a backup route that uses cron to run rsync once a week. I have it output the changed files and errors to a log file. The cron entry to do this is:
0 1 * * 0 rsync -av --delete /home/documents/ /home/backup/ > rsynclog.$(date +%Y%m%d).txt
lets break that down. The first section (0 1 * * 0) tells cron when to do it. In this case every Sunday at 1:00 am. See the cron entry for more info.
Next is the command (rsync..../backup/) see the NAS backup section for more info on this.
The final section creates the log. The v in -av will output which files are changed added or deleted to the backup set. If you do no more, this will be output to the terminal on the screen and will be gone when you close the terminal. To save it to a file we use the >. > redirects the output to the file specified. In our case this is rsynclog.(todays date).txt. Todays date is generated by $(date) and is formatted by +%Y%m%d. See the man page for the date command for more options.
http://www.unixgeeks.org/security/newbie/unix/cron-1.html
http://www.xaprb.com/blog/2006/06/06/what-does-devnull-21-mean/
http://www.cyberciti.biz/faq/linux-log-files-location-and-how-do-i-view-logs-files/
Backing Up
When people talk about servers and backing up, it is important to distinguish between a server that backs up other computers and a method for backing up data on a server. I am doing the latter since I keep all my data on the server anyway. I may look into using the server to back up other computers as my kids get older and I have more computers but for now it is not an issue.
I use a 3 tier backup system. The first tier is the RAID 5 array. This protects against hard disk failure by providing the ability to continue operation if one disk fails without losing any data. Refer to the section on RAID for details.
The second tier is to copy files to backup sets on a completely separate hard drives. This protects against losing two drives at once and provides an older copy of your data in case you accidentally delete something and can't get it back. I use two backup sets on two separate NAS devices (Maxtor Shared Storage II 1TB). One NAS device I keep at work. The other one is at home attached to the network and gets automatically backup up once a week. About once a month I take the weekly one to work and bring the work one home. This is basically an abreviated version of Grandfather Father Son which I call Father and Stepson. See the section on Maxtor NAS for details of this setup.
The third tier as mentioned above is to remove a backup set to a geographically remote area. This protects against theft or fire. You can keep them at work or at a friends house or in a safe deposit box.
Of course there are many ways you can backup your data. You can also look into:
Tower of Hanoi.
Grandfather Father Son
Backup by copying files
Backup by zipping or making a tarball
Backup by making disk images.
sarab
amanda
Disk ARchive (DAR)
Linux Reality episode 63
I use a 3 tier backup system. The first tier is the RAID 5 array. This protects against hard disk failure by providing the ability to continue operation if one disk fails without losing any data. Refer to the section on RAID for details.
The second tier is to copy files to backup sets on a completely separate hard drives. This protects against losing two drives at once and provides an older copy of your data in case you accidentally delete something and can't get it back. I use two backup sets on two separate NAS devices (Maxtor Shared Storage II 1TB). One NAS device I keep at work. The other one is at home attached to the network and gets automatically backup up once a week. About once a month I take the weekly one to work and bring the work one home. This is basically an abreviated version of Grandfather Father Son which I call Father and Stepson. See the section on Maxtor NAS for details of this setup.
The third tier as mentioned above is to remove a backup set to a geographically remote area. This protects against theft or fire. You can keep them at work or at a friends house or in a safe deposit box.
Of course there are many ways you can backup your data. You can also look into:
Tower of Hanoi.
Grandfather Father Son
Backup by copying files
Backup by zipping or making a tarball
Backup by making disk images.
sarab
amanda
Disk ARchive (DAR)
Linux Reality episode 63
SSH and VNC
VNC
VNC is a graphically based remote control program that allows you to control the desktop on a remote computer (the server) from a local computer (the viewer). This is very handy if you want to stick your server in a closet or if you want to be able to access it from the road.
In openSUSE 10.3, open Yast and under Network Devices select Remote Administration (VNC). simply enable it. Modify the sofware firewall to open port 22 if you are using the firewall.
Debian comes all ready to be a VNC server. All you have to do is choose:
Desktop|Preference|Remote Desktop
Then on the configuration page put a check next to:
Allow other users to view your desktop
Allow other users to control your desktop
If you want some security, check
Require the user to enter this password
and enter a password.
VNC is not a secure protocol, so unless you are using SSH tunneling to create a secure connection, your VNC connection is open to anyone on your network. Listen to Linux Reality episode 37 and Linux Reality episode 51 for more information.
SSH
SSH is also a remote control program but it only gives you a command prompt. The advantage to SSH is that is is secure and can be used to make VNC secure.
To set up openSUSE 10.3 as an SSH server, install the OpenSSH package using Yast. Then in the Yast Control Center under System click on System Services (Runlevel) and make sure that sshd has a yes in the enabled column.
To set up Debian as an SSH server, install the package OpenSSH in Synaptic or from the command line type:
#apt-get install ssh openssh-server
In the configuration file you may want to change the port that ssh server listens on. A lot of people will scan for the default ssh port as a way to try and hack into your computer. The default port is 22, but if you are going to open the port on your router, you should make it something different. If you don't open a port on your router, don't worry about it.
To connect to the server (192.168.1.10) from a client linux machine just type
$ssh 192.168.1.10
it will prompt you for username and password
To connect from a windows machine you will need to install an SSH client such as PuTTy.
Open PuTTy and create a saved session with the following
Hostname=192.168.1.10
Port=22 (SSH default) If you changed the SSH port on the server, enter that port instead.
Connection Type=SSH
Connection>Data=your username on the server (if you leave this blank it will ask for your username each time)
type a name for the session and click "Save"
Open the session you just created and log into your server.
VNC over SSH tunnel using TightVNC and Putty
VNC is not a secure connection but SSH is. This will create a secure tunnel using SSH and through that tunnel you can connect with VNC.
Save an SSH session with the following
Hostname=192.168.1.10
Port=22 (SSH default) If you changed the SSH port on the server, enter that port instead.
Connection Type=SSH
Connection>Data=your username on the server
Connection>SSH>tunnels => source port = 5900; destination = 192.168.1.10:5901
session name = vnctunnel
This example is for connecting with the client and server on the same LAN. If you are connecting from a remote location you will need to enter the IP address of your gateway (the WAN side of your router) and you will need to set port forwarding to forward port 22 to your server.
The tunnel will forward port 5900 from the client windows computer (running PuTTy and TightVNC) to port 5901 on the server.
Save the session and connect via SSH.
Open TightVNC viewer on the Windows machine and enter "localhost" for the VNC server.
VNC is a graphically based remote control program that allows you to control the desktop on a remote computer (the server) from a local computer (the viewer). This is very handy if you want to stick your server in a closet or if you want to be able to access it from the road.
In openSUSE 10.3, open Yast and under Network Devices select Remote Administration (VNC). simply enable it. Modify the sofware firewall to open port 22 if you are using the firewall.
Debian comes all ready to be a VNC server. All you have to do is choose:
Desktop|Preference|Remote Desktop
Then on the configuration page put a check next to:
Allow other users to view your desktop
Allow other users to control your desktop
If you want some security, check
Require the user to enter this password
and enter a password.
VNC is not a secure protocol, so unless you are using SSH tunneling to create a secure connection, your VNC connection is open to anyone on your network. Listen to Linux Reality episode 37 and Linux Reality episode 51 for more information.
SSH
SSH is also a remote control program but it only gives you a command prompt. The advantage to SSH is that is is secure and can be used to make VNC secure.
To set up openSUSE 10.3 as an SSH server, install the OpenSSH package using Yast. Then in the Yast Control Center under System click on System Services (Runlevel) and make sure that sshd has a yes in the enabled column.
To set up Debian as an SSH server, install the package OpenSSH in Synaptic or from the command line type:
#apt-get install ssh openssh-server
In the configuration file you may want to change the port that ssh server listens on. A lot of people will scan for the default ssh port as a way to try and hack into your computer. The default port is 22, but if you are going to open the port on your router, you should make it something different. If you don't open a port on your router, don't worry about it.
To connect to the server (192.168.1.10) from a client linux machine just type
$ssh 192.168.1.10
it will prompt you for username and password
To connect from a windows machine you will need to install an SSH client such as PuTTy.
Open PuTTy and create a saved session with the following
Hostname=192.168.1.10
Port=22 (SSH default) If you changed the SSH port on the server, enter that port instead.
Connection Type=SSH
Connection>Data=your username on the server (if you leave this blank it will ask for your username each time)
type a name for the session and click "Save"
Open the session you just created and log into your server.
VNC over SSH tunnel using TightVNC and Putty
VNC is not a secure connection but SSH is. This will create a secure tunnel using SSH and through that tunnel you can connect with VNC.
Save an SSH session with the following
Hostname=192.168.1.10
Port=22 (SSH default) If you changed the SSH port on the server, enter that port instead.
Connection Type=SSH
Connection>Data=your username on the server
Connection>SSH>tunnels => source port = 5900; destination = 192.168.1.10:5901
session name = vnctunnel
This example is for connecting with the client and server on the same LAN. If you are connecting from a remote location you will need to enter the IP address of your gateway (the WAN side of your router) and you will need to set port forwarding to forward port 22 to your server.
The tunnel will forward port 5900 from the client windows computer (running PuTTy and TightVNC) to port 5901 on the server.
Save the session and connect via SSH.
Open TightVNC viewer on the Windows machine and enter "localhost" for the VNC server.
DNS and DHCP
Linux Reality episode 60
Configure dnsmasq
This is all done in the openSUSE operating system
I installed dnsmasq using Yast.
DNS and DHCP are configured using the following files
/etc/hosts
/etc/dnsmasq.conf
/etc/resolv.conf
/etc/hostname
Edit /etc/hosts. This can be done either in Yast via hostnames or directly with a text editor. Add a line for each networked component or computer.
192.168.1.1 router
192.168.1.2 computer1
192.168.1.3 printer
Edit /etc/dnsmasq.conf. Uncomment and edit the following lines
expand-hosts
domain=yourdomain.lan
dhcp-range=192.168.1.100,192.168.1.150,168h
dhcp-option=3,192.168.1.1
expand-hosts will automatically add .yourdomain.lan to hostnames in /etc/hosts
yourdomain can be called whatever you want. The value has to match what is shown in Yast under hostname and domain.
dhcp-range specifies the range of ip addresses available for dhcp. The last entry (168h) specifies the duration of the lease. It can be entered as minutes (m), hours (h) or infinite (infinite).
dhcp-option=3 specifies the router (gateway) ip address. If you don't change this the default is to use the server ip as the router. If you have a Linksys or other router and have not configured the server as a router you will not be able to get external ip addresses or websites.
/etc/resolve.conf on the dns/dhcp server should only contain the external name servers. On my machine these are:
nameserver 66.75.160.63
nameserver 66.75.160.64
/etc/hostname should contain the hostname and domain. On my machine this is:
server.yourdomain.lan
to verify the hostname is what you think it is, type
$hostname
this will return the hostname without the domain. On my machine this is:
server
In Yast open the runtime services GUI and enable dnsmasq to startup at boot.
On the command line restart dnsmasq
#/etc/init.d/dnsmasq restart
Now you can open a browser and type
printer.yourdomain.lan
and the web configuration screen for your printer will appear. Or,
router.yourdomain.lan
and the configuration screen for your router will appear.
In the mail client enter
server.yourdomain.lan
to point to the IMAP server
Don't forget to update your network components to use the server ip address for dns or dhcp and to disable dhcp on your router.
Configure dnsmasq
This is all done in the openSUSE operating system
I installed dnsmasq using Yast.
DNS and DHCP are configured using the following files
/etc/hosts
/etc/dnsmasq.conf
/etc/resolv.conf
/etc/hostname
Edit /etc/hosts. This can be done either in Yast via hostnames or directly with a text editor. Add a line for each networked component or computer.
192.168.1.1 router
192.168.1.2 computer1
192.168.1.3 printer
Edit /etc/dnsmasq.conf. Uncomment and edit the following lines
expand-hosts
domain=yourdomain.lan
dhcp-range=192.168.1.100,192.168.1.150,168h
dhcp-option=3,192.168.1.1
expand-hosts will automatically add .yourdomain.lan to hostnames in /etc/hosts
yourdomain can be called whatever you want. The value has to match what is shown in Yast under hostname and domain.
dhcp-range specifies the range of ip addresses available for dhcp. The last entry (168h) specifies the duration of the lease. It can be entered as minutes (m), hours (h) or infinite (infinite).
dhcp-option=3 specifies the router (gateway) ip address. If you don't change this the default is to use the server ip as the router. If you have a Linksys or other router and have not configured the server as a router you will not be able to get external ip addresses or websites.
/etc/resolve.conf on the dns/dhcp server should only contain the external name servers. On my machine these are:
nameserver 66.75.160.63
nameserver 66.75.160.64
/etc/hostname should contain the hostname and domain. On my machine this is:
server.yourdomain.lan
to verify the hostname is what you think it is, type
$hostname
this will return the hostname without the domain. On my machine this is:
server
In Yast open the runtime services GUI and enable dnsmasq to startup at boot.
On the command line restart dnsmasq
#/etc/init.d/dnsmasq restart
Now you can open a browser and type
printer.yourdomain.lan
and the web configuration screen for your printer will appear. Or,
router.yourdomain.lan
and the configuration screen for your router will appear.
In the mail client enter
server.yourdomain.lan
to point to the IMAP server
Don't forget to update your network components to use the server ip address for dns or dhcp and to disable dhcp on your router.
IMAP Mail Server
An IMAP mail server stores your email on a server and allows you to view it over a network on any computer with an email client that is configured to access the IMAP server. All of your email, including sent items and drafts is available from any computer. If you have more than one computer in your household this can be very handy. You could send an email from your wife's computer, then go back to your computer and it would be there.
The basic idea is that you have two programs. The first gets the mail from your POP3 accounts (yahoo, gmail, hotmail, etc) and saves them in a local directory. Emails in their raw form are just text. Attachments are encoded as text as well.
A second program looks at the email files and makes them available to email clients using the IMAP protocol. They are not copied to the client, although they can be synchronized if you desire, so no matter what computer you use to access your email, it will be the same. You can conceivably use this to implement remote access to your email, but I have not done that yet.
I use the setup recommended by Linux Reality episode 61. This uses Getmail to get the mail and Dovecot as the IMAP server. Dovecot can serve IMAP from either maildir or mbox format. Basically, maildir creates a directory structure and saves each email as separate file whereas mbox creates a single file with all of your email messages concatenated together. This makes some operations slower and can create file access problems. I am using maildir.
GETMAIL
Getmail will not create the maildir folders, so you will need to manually create the following four folders:
/home/user/Maildir
/home/user/Maildir/new
/home/user/Maildir/cur
/home/user/Maildir/tmp
Installing getmail in Debian is easy. You can either use Synaptic or from the command line:
#apt-get install getmail4
Setting up getmail is very easy. You create a config file called getmailrc and put it in the following folder: /home/user/.getmail/getmailrc
This file contains two sections. The first tells getmail where to get mail from and the second tells getmail where to put the mail. Here is my getmailrc file.
**********
#This is the getmail configuration file for my-server
#01 October 2007
#for a complete, commented file, see http://pyropus.ca/software/getmail/configuration.html
#this file should be located in /home/user/.getmail
[retriever]
type = SimplePOP3SSLRetriever
server = pop.mail.yahoo.com
username = username
port = 995
password = password
use_apop = False
[destination]
type = Maildir
path = ~/Maildir/
[options]
verbose = 1
read_all = true
delete = true
message_log = ~/.getmail/log
**********
Getmail does not run automatically (as a daemon). To execute getmail you use the command:
$getmail
Getmail will run once and then it is done.
To automate getmail, you have to create a cron job. This is discussed in Linux Reality episode 39 with the basic steps spelled out below
Login as the user you want to run the cron job.
$crontab -e
The basic format for a crontab entry is as follows (separated by spaces)
minutes hours dayofmonth month dayofweek command
minutes range is 0-59
hours range is 0-23
dayofmonth range is 1-31
month range is 1-12
day of week range is 0-6, 0=Sunday
In order to for getmail to check mail every minute, add the following line:
* * * * * getmail
or, to check email every hour on the hour, add the following line
0 * * * * getmail
save and exit
I have had no problems running getmail for my Yahoo account. For my gmail account I have had an issue where gmail was hiding messages that had been read or downloaded but not deleted. To fix this open gmail and then under Setting choose Forwarding and POP/IMAP and select "Enable POP for all mail (even mail that's already been downloaded)". This will download all messages and then revert to the default setting.
DOVECOT
The first time I tried to install dovecot it failed completely. I never found out what was wrong and abandoned it. I searched the internet and came up with courier as an alternative. I installed courier and used it successfully for a while until one day I was editing the config file and the whole thing crapped out. I couldn't uninstall it, I couldn't fix it, I couldn't do anything. I ended up reformatting the hard drive and rebuilding the whole server. I installed dovecot the second time and it works great.
Installing dovecot is simple using Synaptic or simply type the command:
#apt-get install dovecot???
Configuring dovecot is simply a matter of editing the following file:
/etc/dovecot/dovecot.conf
Below are the lines that I had to edit in my dovecot.conf file. Note that I am using dovecot behind a NAT firewall where I have absolutely no risk of exposure so I have enabled plain text authorization. If your network is not secure do not do this.
**********
protocols = imap #imaps
disable_plaintext_auth = no
login_greeting = Dovecot ready on my-server
mail_location = maildir:/home/user/Maildir
**********
The basic idea is that you have two programs. The first gets the mail from your POP3 accounts (yahoo, gmail, hotmail, etc) and saves them in a local directory. Emails in their raw form are just text. Attachments are encoded as text as well.
A second program looks at the email files and makes them available to email clients using the IMAP protocol. They are not copied to the client, although they can be synchronized if you desire, so no matter what computer you use to access your email, it will be the same. You can conceivably use this to implement remote access to your email, but I have not done that yet.
I use the setup recommended by Linux Reality episode 61. This uses Getmail to get the mail and Dovecot as the IMAP server. Dovecot can serve IMAP from either maildir or mbox format. Basically, maildir creates a directory structure and saves each email as separate file whereas mbox creates a single file with all of your email messages concatenated together. This makes some operations slower and can create file access problems. I am using maildir.
GETMAIL
Getmail will not create the maildir folders, so you will need to manually create the following four folders:
/home/user/Maildir
/home/user/Maildir/new
/home/user/Maildir/cur
/home/user/Maildir/tmp
Installing getmail in Debian is easy. You can either use Synaptic or from the command line:
#apt-get install getmail4
Setting up getmail is very easy. You create a config file called getmailrc and put it in the following folder: /home/user/.getmail/getmailrc
This file contains two sections. The first tells getmail where to get mail from and the second tells getmail where to put the mail. Here is my getmailrc file.
**********
#This is the getmail configuration file for my-server
#01 October 2007
#for a complete, commented file, see http://pyropus.ca/software/getmail/configuration.html
#this file should be located in /home/user/.getmail
[retriever]
type = SimplePOP3SSLRetriever
server = pop.mail.yahoo.com
username = username
port = 995
password = password
use_apop = False
[destination]
type = Maildir
path = ~/Maildir/
[options]
verbose = 1
read_all = true
delete = true
message_log = ~/.getmail/log
**********
Getmail does not run automatically (as a daemon). To execute getmail you use the command:
$getmail
Getmail will run once and then it is done.
To automate getmail, you have to create a cron job. This is discussed in Linux Reality episode 39 with the basic steps spelled out below
Login as the user you want to run the cron job.
$crontab -e
The basic format for a crontab entry is as follows (separated by spaces)
minutes hours dayofmonth month dayofweek command
minutes range is 0-59
hours range is 0-23
dayofmonth range is 1-31
month range is 1-12
day of week range is 0-6, 0=Sunday
In order to for getmail to check mail every minute, add the following line:
* * * * * getmail
or, to check email every hour on the hour, add the following line
0 * * * * getmail
save and exit
I have had no problems running getmail for my Yahoo account. For my gmail account I have had an issue where gmail was hiding messages that had been read or downloaded but not deleted. To fix this open gmail and then under Setting choose Forwarding and POP/IMAP and select "Enable POP for all mail (even mail that's already been downloaded)". This will download all messages and then revert to the default setting.
DOVECOT
The first time I tried to install dovecot it failed completely. I never found out what was wrong and abandoned it. I searched the internet and came up with courier as an alternative. I installed courier and used it successfully for a while until one day I was editing the config file and the whole thing crapped out. I couldn't uninstall it, I couldn't fix it, I couldn't do anything. I ended up reformatting the hard drive and rebuilding the whole server. I installed dovecot the second time and it works great.
Installing dovecot is simple using Synaptic or simply type the command:
#apt-get install dovecot???
Configuring dovecot is simply a matter of editing the following file:
/etc/dovecot/dovecot.conf
Below are the lines that I had to edit in my dovecot.conf file. Note that I am using dovecot behind a NAT firewall where I have absolutely no risk of exposure so I have enabled plain text authorization. If your network is not secure do not do this.
**********
protocols = imap #imaps
disable_plaintext_auth = no
login_greeting = Dovecot ready on my-server
mail_location = maildir:/home/user/Maildir
**********
Media Server
TwonkyMedia is a UPnP AV media server that I use to serve music, pictures and video to a D-Link DSM-520 media server. I am very happy with the way it works.
In order to get TwonkyMedia to work with my AMD64 processor, I had to install a package to give Debian 32 bit program support. This package is ia32-libs and can be installed in Synaptic or on the command line by typing:
#apt-get install ia32-libs
I used the script from the TwonkyMedia website to install the software. I followed the directions, keeping all defaults. The only issue I had was that I had to correct file permissions twice when I got error messages saying the scripts could not be run.
Once it is installed, using a web browser, go to http://server_IP_address:9000 to configure TwonkyMedia.
In order to get TwonkyMedia to work with my AMD64 processor, I had to install a package to give Debian 32 bit program support. This package is ia32-libs and can be installed in Synaptic or on the command line by typing:
#apt-get install ia32-libs
I used the script from the TwonkyMedia website to install the software. I followed the directions, keeping all defaults. The only issue I had was that I had to correct file permissions twice when I got error messages saying the scripts could not be run.
Once it is installed, using a web browser, go to http://server_IP_address:9000 to configure TwonkyMedia.
Music Server
I had Firefly (mt-daap) working great on Debian, but cannot make it work on openSUSE. I installed tangerine instead. It doesn't have some of the features of mt-daap, like smart playlists, so I will probably try mt-daap again in the future.
Tangerine was very easy to set up in openSUSE. I installed it using Yast and then it showed up in the Yast configuration screen. You get the GUI configuration screen shown on their website and you are off and running.
Firefly music server is a daap server that used to be called mt-daap. I will call it mt-daap here because it is easier. Daap is the protocol that iTunes uses to share its media library. I use mt-daap to share my music collection on my home network without having to keep iTunes running all the time (not to mention that there is no Linux version of iTunes). Mt-daap is very small and very fast.
To install mt-daap in Debian, simply select it in Synaptic or on the command line type:
#apt-get install mt-daapd
To configure mt-daap you will need to edit the mp3_dir line in /etc/mt-daap.conf to point to your music collection. You may also want to change the admin password, but that is not necessary.
Tangerine was very easy to set up in openSUSE. I installed it using Yast and then it showed up in the Yast configuration screen. You get the GUI configuration screen shown on their website and you are off and running.
Firefly music server is a daap server that used to be called mt-daap. I will call it mt-daap here because it is easier. Daap is the protocol that iTunes uses to share its media library. I use mt-daap to share my music collection on my home network without having to keep iTunes running all the time (not to mention that there is no Linux version of iTunes). Mt-daap is very small and very fast.
To install mt-daap in Debian, simply select it in Synaptic or on the command line type:
#apt-get install mt-daapd
To configure mt-daap you will need to edit the mp3_dir line in /etc/mt-daap.conf to point to your music collection. You may also want to change the admin password, but that is not necessary.
RAID
I had a lot of trouble getting a RAID card to work. It finally worked because 3ware RAID cards are supported in Linux kernel 2.6.19 or later. I had to switch the distribution I was using from Debian Etch to openSUSE 10.3 because Debian Etch uses 2.6.18.
I finally got RAID to work using openSUSE 10.3 and a 3ware 9650EX-4LPML RAID card. I am using four 400 GB SATA hard drives, one from Western Digital and three from Seagate. If you are using larger hard drives you will need to mount the array after you have installed the operating systems as the installer does not support volumes larger than 2.0 GB. Check out the section on AutoCarving in the user manaul for details. While I detail my difficulties below, I will start with the success. This description assumes that you have a separate hard drive for the OS and will mount the RAID array as /home.
The simplest way to do this is to set up RAID when you install the operating system. Install the card and 4 hard drives. When the RAID BIOS is recognized onscreen, hit alt+3 to enter RAID setup. Follow the prompts to create a RAID 5 logical volume using all four drives. You will then need to initialize the array. This will take a long time.
When the array is initialized, save and exit the BIOS screen and continue booting from the installation disk. Follow the prompts and when when it gets the disc usage part select the single hard drive to install / and pick the RAID array to install /home. Complete the OS installation and you are done. Your array is then seamlessly integrated as /home which is where you store all your data.
The array will appear in /dev as /dev/sdb and the volume as /dev/sdb1. The hard drive with the OS is /dev/sda and the volume is /dev/sda1.
If you do not set up the array during installation, you can add it after the fact using the command line. I was not able to create a volume of format the array using GParted. I had to use fdisk and mkfs. It went like this.
#fdisk /dev/sdb
type n for a new volume. accept the defaults for the start and finish. This will use the entire array as a single volume.
type t to change the type and select 83, which is a Linux partition.
p will print the partition table to the screen. It should look like this
Device Boot Start End Blocks ID System
/dev/sdb1 1 4088 2060320 83 Linux
w will save the changes. Note that you will need to reboot before the new partition table will be used so don't try to mount the new volume until after you have rebooted. But don't reboot yet.
#mkfs.ext3 /dev/sdb1
this will format the array with ext3. This will take a while
When it is done, reboot then
#gedit /etc/fstab
add the following line
/dev/sdb1 /home ext3 defaults 1 2
then
#mount -a
you're done.
Some of the references here came from this article on software RAID.
*********************************************
The rest of this describes the long string of failures I had
*********************************************
I bought a HighPoint RocketRAID 2300 and downloaded the latest BIOS/Driver from their website. I am using kernel 2.6.18.5. The website states that you can use 2.4.x or 2.6.x kernels, but when I try to "make", I get the error message
expr: syntax error
../../../inc/linux/Makefile.def:85: *** Only kernel 2.4/2.6 is supported but you use 2.. Stop.
Doing some research at I came across this list of RAID cards with comments. Based on the reference to the Promise EX8350, I bought the Promise EX4350 which is supposed to be the same thing but with 4 ports in stead of 8.
A discussion of RAID levels can be found at Wikipedia.
Debian x386 device database Promise EX4350
uses stex module
PCI ID = 105a:4302
13 January 2008
The basic steps for setting up the Promise RAID array are:
1. Install the controller card and hard drives
2. Create a logical volume.
3. Compile the driver.
4. Set up WebPAM
I installed the card and 3 hard drives and created a logical volume. Each disk is 400 GB, so using RAID 5 the volume has 800 GB capacity. I plan to add a fourth drive later, but I need to get a 5.25 to 3.5 bay adapter. The case claims to hold 6 3.5 drives, but two of them are floppy sized. The floppy bays are smaller and provide less room for airflow, so I decided to stick the fifth drive in the unused 5.25 in bay.
I ran into a problem when I tried to compile the driver however. The driver is compiled as a kernel patch and this required me to download the kernel source code (luckily it is in Synaptic (linux-source-2.6.18)). It then uses the kernel configuration information to configure the driver. I got an error that stated that the configuration file did not match my kernel configuration.
Next step. Try and find out what my kernel configuration is. I wonder if I would be having this problem if I installed from a full set of disks instead of the NetInstall.
When in installed the linux source code from Synaptic, it but a tar ball on my hard drive at
/usr/src/linux-source-2.6.18.tar.bz2
next run the command
# bzip2 -cd linux-source-2.6.18.tar.bz2 | tar xvf -
this will make a folder
/usr/src/linux-source-2.6.18
with all the source code files in it
Next, go to that directory as root
#cd /usr/src/linux-source-2.6.18
#make mrproper
#make oldconfig
#make prepare
#make modules_prepare
Next, download the driver source file from www.promise.com
(http://www.promise.com/upload/Support/Driver/shasta_linux_src_2.9.0.22.tgz)
and unzip it to a directory
/home/user/Desktop/Downloads/2.9.0.22
This directory will contain 3 files, Makefile, Readme and stex.c
Make a backup copy of Makefile then open it with a Linux based text editor (do not use notepad) and change the kernel source directory line from...
################# kernel configuration ###############
ifeq ("x", "x$(KERNEL_SOURCE_DIR)")
KERNEL_SRC_DIR=$(shell if ls /lib/modules/`uname -r`/build >/dev/null 2>&1; then echo /lib/modules/`uname -r`/build; elif ls /lib/modules/`uname -r`/source >/dev/null 2>&1; then echo /lib/modules/`uname -r`/source ;elif ls /usr/src/linux-2.6 >/dev/null 2>&1; then echo /usr/src/linux-2.6; else echo /usr/src/linux; fi)
else
KERNEL_SRC_DIR=$(KERNEL_SOURCE_DIR)
endif
to...
################# kernel configuration ###############
ifeq ("x", "x$(KERNEL_SOURCE_DIR)")
KERNEL_SRC_DIR=$(shell if ls /lib/modules/`uname -r`/build >/dev/null 2>&1; then echo /lib/modules/`uname -r`/build; elif ls /lib/modules/`uname -r`/source >/dev/null 2>&1; then echo /lib/modules/`uname -r`/source ;elif ls /usr/src/linux-2.6 >/dev/null 2>&1; then echo /usr/src/linux-2.6; else echo /usr/src/linux-source-2.6.18; fi)
else
KERNEL_SRC_DIR=$(KERNEL_SOURCE_DIR)
endif
Next, go to the directory with the driver source
#cd /home/user/Desktop/Downloads/2.9.0.22
#make clean
#make KERNEL_SOURCE_DIR=$KERN_SRC_DIR
if everything worked, this will take a while. It took about 20 minutes for me.
This will make a file called stex.ko. The final step is to install this driver file.
#insmod stex.ko
I did all of this but could not get the array to show up anywhere I could do anything with it. In retrospect maybe all I needed to do was mount the array, but I did not think of it at the time and don't know if it would actually have solved all my problems. At any rate I tried following the directions to get WebPAM working and got nowhere. At this point I gave up on hardware RAID and decided to use software RAID.
Following the instructions at this website, Setting up Linux software RAID, I made four volumes from one hard drive since I only had two SATA ports on the motherboard. I was able to do everything successfully so I went and bought a PCIe to SATA controller card only to discover that an amateur Linux user such as myself couldn't make that work either. I needed something that would work out of the box with minimal configuration. Compiling kernel drivers was stretching my knowledge and my patience.
My research led me to 3ware. I bought the 3ware 9650SE-4lpl card. It turns out that 3ware drivers part of the Linux kernel version 2.6.19 or later. Since Debian Etch is on 2.6.18 I went looking for a way to get 3ware drivers on Debian. I found a Debian Etch ISO created by 3ware that incorporated the drivers I needed, but could not get it to install. I tried Debian Lenny (testing) but could not get it to install either. My patience had grown short and I did not give either of the options much time. I had recieved a copy of Windows server 2003 and was getting closer and closer to ditching Linux and using that. Only my contempt for windows kept me from jumping ship.
I finally decided to abandon Debian and go to one of the distributions that was explicitly supported in the driver and had a 2.6.19 or greater kernel. The answer ended up being openSUSE 10.3. I installed the 3ware card and 3 drives, then installed the openSUSE OS. It recognized the array right away and allowed me to mount it as /home during installation. I was finally on my way. Details of this successful installation are in the RAID that did work chapter.
I finally got RAID to work using openSUSE 10.3 and a 3ware 9650EX-4LPML RAID card. I am using four 400 GB SATA hard drives, one from Western Digital and three from Seagate. If you are using larger hard drives you will need to mount the array after you have installed the operating systems as the installer does not support volumes larger than 2.0 GB. Check out the section on AutoCarving in the user manaul for details. While I detail my difficulties below, I will start with the success. This description assumes that you have a separate hard drive for the OS and will mount the RAID array as /home.
The simplest way to do this is to set up RAID when you install the operating system. Install the card and 4 hard drives. When the RAID BIOS is recognized onscreen, hit alt+3 to enter RAID setup. Follow the prompts to create a RAID 5 logical volume using all four drives. You will then need to initialize the array. This will take a long time.
When the array is initialized, save and exit the BIOS screen and continue booting from the installation disk. Follow the prompts and when when it gets the disc usage part select the single hard drive to install / and pick the RAID array to install /home. Complete the OS installation and you are done. Your array is then seamlessly integrated as /home which is where you store all your data.
The array will appear in /dev as /dev/sdb and the volume as /dev/sdb1. The hard drive with the OS is /dev/sda and the volume is /dev/sda1.
If you do not set up the array during installation, you can add it after the fact using the command line. I was not able to create a volume of format the array using GParted. I had to use fdisk and mkfs. It went like this.
#fdisk /dev/sdb
type n for a new volume. accept the defaults for the start and finish. This will use the entire array as a single volume.
type t to change the type and select 83, which is a Linux partition.
p will print the partition table to the screen. It should look like this
Device Boot Start End Blocks ID System
/dev/sdb1 1 4088 2060320 83 Linux
w will save the changes. Note that you will need to reboot before the new partition table will be used so don't try to mount the new volume until after you have rebooted. But don't reboot yet.
#mkfs.ext3 /dev/sdb1
this will format the array with ext3. This will take a while
When it is done, reboot then
#gedit /etc/fstab
add the following line
/dev/sdb1 /home ext3 defaults 1 2
then
#mount -a
you're done.
Some of the references here came from this article on software RAID.
*********************************************
The rest of this describes the long string of failures I had
*********************************************
I bought a HighPoint RocketRAID 2300 and downloaded the latest BIOS/Driver from their website. I am using kernel 2.6.18.5. The website states that you can use 2.4.x or 2.6.x kernels, but when I try to "make", I get the error message
expr: syntax error
../../../inc/linux/Makefile.def:85: *** Only kernel 2.4/2.6 is supported but you use 2.. Stop.
Doing some research at I came across this list of RAID cards with comments. Based on the reference to the Promise EX8350, I bought the Promise EX4350 which is supposed to be the same thing but with 4 ports in stead of 8.
A discussion of RAID levels can be found at Wikipedia.
Debian x386 device database Promise EX4350
uses stex module
PCI ID = 105a:4302
13 January 2008
The basic steps for setting up the Promise RAID array are:
1. Install the controller card and hard drives
2. Create a logical volume.
3. Compile the driver.
4. Set up WebPAM
I installed the card and 3 hard drives and created a logical volume. Each disk is 400 GB, so using RAID 5 the volume has 800 GB capacity. I plan to add a fourth drive later, but I need to get a 5.25 to 3.5 bay adapter. The case claims to hold 6 3.5 drives, but two of them are floppy sized. The floppy bays are smaller and provide less room for airflow, so I decided to stick the fifth drive in the unused 5.25 in bay.
I ran into a problem when I tried to compile the driver however. The driver is compiled as a kernel patch and this required me to download the kernel source code (luckily it is in Synaptic (linux-source-2.6.18)). It then uses the kernel configuration information to configure the driver. I got an error that stated that the configuration file did not match my kernel configuration.
Next step. Try and find out what my kernel configuration is. I wonder if I would be having this problem if I installed from a full set of disks instead of the NetInstall.
When in installed the linux source code from Synaptic, it but a tar ball on my hard drive at
/usr/src/linux-source-2.6.18.tar.bz2
next run the command
# bzip2 -cd linux-source-2.6.18.tar.bz2 | tar xvf -
this will make a folder
/usr/src/linux-source-2.6.18
with all the source code files in it
Next, go to that directory as root
#cd /usr/src/linux-source-2.6.18
#make mrproper
#make oldconfig
#make prepare
#make modules_prepare
Next, download the driver source file from www.promise.com
(http://www.promise.com/upload/Support/Driver/shasta_linux_src_2.9.0.22.tgz)
and unzip it to a directory
/home/user/Desktop/Downloads/2.9.0.22
This directory will contain 3 files, Makefile, Readme and stex.c
Make a backup copy of Makefile then open it with a Linux based text editor (do not use notepad) and change the kernel source directory line from...
################# kernel configuration ###############
ifeq ("x", "x$(KERNEL_SOURCE_DIR)")
KERNEL_SRC_DIR=$(shell if ls /lib/modules/`uname -r`/build >/dev/null 2>&1; then echo /lib/modules/`uname -r`/build; elif ls /lib/modules/`uname -r`/source >/dev/null 2>&1; then echo /lib/modules/`uname -r`/source ;elif ls /usr/src/linux-2.6 >/dev/null 2>&1; then echo /usr/src/linux-2.6; else echo /usr/src/linux; fi)
else
KERNEL_SRC_DIR=$(KERNEL_SOURCE_DIR)
endif
to...
################# kernel configuration ###############
ifeq ("x", "x$(KERNEL_SOURCE_DIR)")
KERNEL_SRC_DIR=$(shell if ls /lib/modules/`uname -r`/build >/dev/null 2>&1; then echo /lib/modules/`uname -r`/build; elif ls /lib/modules/`uname -r`/source >/dev/null 2>&1; then echo /lib/modules/`uname -r`/source ;elif ls /usr/src/linux-2.6 >/dev/null 2>&1; then echo /usr/src/linux-2.6; else echo /usr/src/linux-source-2.6.18; fi)
else
KERNEL_SRC_DIR=$(KERNEL_SOURCE_DIR)
endif
Next, go to the directory with the driver source
#cd /home/user/Desktop/Downloads/2.9.0.22
#make clean
#make KERNEL_SOURCE_DIR=$KERN_SRC_DIR
if everything worked, this will take a while. It took about 20 minutes for me.
This will make a file called stex.ko. The final step is to install this driver file.
#insmod stex.ko
I did all of this but could not get the array to show up anywhere I could do anything with it. In retrospect maybe all I needed to do was mount the array, but I did not think of it at the time and don't know if it would actually have solved all my problems. At any rate I tried following the directions to get WebPAM working and got nowhere. At this point I gave up on hardware RAID and decided to use software RAID.
Following the instructions at this website, Setting up Linux software RAID, I made four volumes from one hard drive since I only had two SATA ports on the motherboard. I was able to do everything successfully so I went and bought a PCIe to SATA controller card only to discover that an amateur Linux user such as myself couldn't make that work either. I needed something that would work out of the box with minimal configuration. Compiling kernel drivers was stretching my knowledge and my patience.
My research led me to 3ware. I bought the 3ware 9650SE-4lpl card. It turns out that 3ware drivers part of the Linux kernel version 2.6.19 or later. Since Debian Etch is on 2.6.18 I went looking for a way to get 3ware drivers on Debian. I found a Debian Etch ISO created by 3ware that incorporated the drivers I needed, but could not get it to install. I tried Debian Lenny (testing) but could not get it to install either. My patience had grown short and I did not give either of the options much time. I had recieved a copy of Windows server 2003 and was getting closer and closer to ditching Linux and using that. Only my contempt for windows kept me from jumping ship.
I finally decided to abandon Debian and go to one of the distributions that was explicitly supported in the driver and had a 2.6.19 or greater kernel. The answer ended up being openSUSE 10.3. I installed the 3ware card and 3 drives, then installed the openSUSE OS. It recognized the array right away and allowed me to mount it as /home during installation. I was finally on my way. Details of this successful installation are in the RAID that did work chapter.
File Server
Samba is the open source version of the SMB protocol that is used by Windows for file sharing. It is also supported by Mac and so is the best choice for a mixed network. A discussion of setting up Samba in Debian can be found in the following episodes of Linux Reality.
The key to making Samba work with Windows is to have common usernames and passwords for Windows, Linux and Samba. This is described in detail in Linux Reality episode 26.
Create a Windows user with username "bob" and password "tom"
Create a Linux user using Yast with username "bob and password "tom"
Create a Samba user as follows
To add a Samba user called "bob"
# smbpasswd -a bob
You will then be prompted for a password. Enter "tom" twice.
Access a Samba share from Linux
Linux Reality Episode 25
Access a Linux Samba share from Windows
Linux Reality Episode 26
CUPS print server and Samba file server
Linux Reality Episode 59
The next step is to create a share on the Linux server.
When I switched to openSUSE (see the RAID chapter) things became much simpler because of YAST. In the Samba configuration GUI in I changed the workgroup to HOME, which is the name of my windows workgroup, then added a share called "files", accepting all of the defaults.
Then from the windows machine I mapped a new network drive and browsed to "files". It connected and works great.
The one glitch I had was when I first created the share, I called it "documents". Then when I tried to copy files to it from windows I got an error message saying it was read only. I have a shared folder on my windows machine with the same name, so that may be the problem or it could be something else.
The key to making Samba work with Windows is to have common usernames and passwords for Windows, Linux and Samba. This is described in detail in Linux Reality episode 26.
Create a Windows user with username "bob" and password "tom"
Create a Linux user using Yast with username "bob and password "tom"
Create a Samba user as follows
To add a Samba user called "bob"
# smbpasswd -a bob
You will then be prompted for a password. Enter "tom" twice.
Access a Samba share from Linux
Linux Reality Episode 25
Access a Linux Samba share from Windows
Linux Reality Episode 26
CUPS print server and Samba file server
Linux Reality Episode 59
The next step is to create a share on the Linux server.
When I switched to openSUSE (see the RAID chapter) things became much simpler because of YAST. In the Samba configuration GUI in I changed the workgroup to HOME, which is the name of my windows workgroup, then added a share called "files", accepting all of the defaults.
Then from the windows machine I mapped a new network drive and browsed to "files". It connected and works great.
The one glitch I had was when I first created the share, I called it "documents". Then when I tried to copy files to it from windows I got an error message saying it was read only. I have a shared folder on my windows machine with the same name, so that may be the problem or it could be something else.
Web Server
The following are excellent references for setting up a home web server.
Apache web server
Linux Reality Episode 56
www.lifehacker.com set up a home web server
Debian comes with Apache 1.3 and 2.2. In the package manager Apache 1.3 is called apache and Apache 2.2 is called apache2.
INSTALL APACHE
Install this using Synaptic or from the command line type:
#apt-get install apache
CONFIGURE APACHE
Go to the configuration file directory
#cd /etc/apache2
Create a backup of the configuration file
#cp httpd.conf httpd.conf_backup
#cp default-server.conf default-server.conf_backup
Edit the configuration file
#nano httpd.conf
In the Global Environment section
add index.htm to the DirectoyIndex line
This will make it more user friendly if you are creating web pages in windows.
save and exit
#nano default-server.conf
DocumentRoot defines where the web pages are stored. The default is
DocumentRoot /var/www
I like to keep all of my documents in my home directory to simplify backup, but you can keep yours wherever you like. For this example, lets call my Document Root
/home/docs/website
A bit further down is a line that looks like
<"Directory /var/www"> (without the quotes)
Change this to whatever you made the DocumentRoot. Leave the trailing /.
A few lines down change
AllowOverride None
to
AllowOverride All
This will allow you to put a password on your website. If you don't want a password, you don't need to change this, but I would anyway.
Save the file (cntrl+O) and exit (cntrl+X)
PASSWORD PROTECT YOUR WEBSITE
In the terminal type:
#htpasswd -c /home/user/passwords username
This will create a password file called "passwords" in the folder /home/user
Replace both the path and the filename with whatever you want to use.
Replace username with the username you want people to log into your website with.You can have multiple logins by adding usernames to this file. Type "$man htpasswd" for more info.
Next you will need to create a file in the DocumentRoot directory that directs logins to your password file.
Go to the DocumentRoot directory
$cd /home/docs/website
Create a file called .htaccess
$touch .htaccess
Open the .htaccess file for editing
$nano .htaccess
Type or copy the following into the .htaccess file
*********
AuthType Basic
AuthName "name of your website"
AuthUserFile /home/user/passwords
AuthGroupFile /dev/null
require valid-user
**********
OPEN YOUR WEBSITE TO THE WORLD
The last two things to do if you want to share your website with the world are to forward port 80 from your router to your server and get a domain name for your website. Excellent instructions on how to do this can be found at
http://lifehacker.com/software/top/geek-to-live--how-to-access-a-home-server-behind-a-routerfirewall-127276.php
http://lifehacker.com/software/web-publishing/geek-to-live--how-to-assign-a-domain-name-to-your-home-web-server-124804.php
Apache for openSUSE
Apache web server
Linux Reality Episode 56
www.lifehacker.com set up a home web server
Debian comes with Apache 1.3 and 2.2. In the package manager Apache 1.3 is called apache and Apache 2.2 is called apache2.
INSTALL APACHE
Install this using Synaptic or from the command line type:
#apt-get install apache
CONFIGURE APACHE
Go to the configuration file directory
#cd /etc/apache2
Create a backup of the configuration file
#cp httpd.conf httpd.conf_backup
#cp default-server.conf default-server.conf_backup
Edit the configuration file
#nano httpd.conf
In the Global Environment section
add index.htm to the DirectoyIndex line
This will make it more user friendly if you are creating web pages in windows.
save and exit
#nano default-server.conf
DocumentRoot defines where the web pages are stored. The default is
DocumentRoot /var/www
I like to keep all of my documents in my home directory to simplify backup, but you can keep yours wherever you like. For this example, lets call my Document Root
/home/docs/website
A bit further down is a line that looks like
<"Directory /var/www"> (without the quotes)
Change this to whatever you made the DocumentRoot. Leave the trailing /.
A few lines down change
AllowOverride None
to
AllowOverride All
This will allow you to put a password on your website. If you don't want a password, you don't need to change this, but I would anyway.
Save the file (cntrl+O) and exit (cntrl+X)
PASSWORD PROTECT YOUR WEBSITE
In the terminal type:
#htpasswd -c /home/user/passwords username
This will create a password file called "passwords" in the folder /home/user
Replace both the path and the filename with whatever you want to use.
Replace username with the username you want people to log into your website with.
Go to the DocumentRoot directory
$cd /home/docs/website
Create a file called .htaccess
$touch .htaccess
Open the .htaccess file for editing
$nano .htaccess
Type or copy the following into the .htaccess file
*********
AuthType Basic
AuthName "name of your website"
AuthUserFile /home/user/passwords
AuthGroupFile /dev/null
require valid-user
**********
OPEN YOUR WEBSITE TO THE WORLD
The last two things to do if you want to share your website with the world are to forward port 80 from your router to your server and get a domain name for your website. Excellent instructions on how to do this can be found at
http://lifehacker.com/software/top/geek-to-live--how-to-access-a-home-server-behind-a-routerfirewall-127276.php
http://lifehacker.com/software/web-publishing/geek-to-live--how-to-assign-a-domain-name-to-your-home-web-server-124804.php
Apache for openSUSE
Operating System
openSUSE 10.3
I started this project with Debian Etch as described below, but ended up switching to openSUSE 10.3 for support of my RAID card. When Debian Lenny become the stable release, I might look at switching back to Debian since I like their command line functionality better. Yast is nice, but sometimes I am not sure exactly what it is doing and it is a bit slower than Debian once you get used to the command line. OpenSUSE 10.3 was my second choice primarily because it used a Linux kernel >2.6.19, and secondly because it was mentioned in most of the RAID hardware literature.
Installing SUSE was easy. I downloaded the DVD ISO image from www.opensuse.org and burned it to a DVD. When I started the installation, the RAID card and drives were installed and a volume had already been created. It was pretty easy during installation to specify that the file structure be mounted on the OS drive with the exception of /home, which was mounted on the RAID array. The menus were all intuitive and easy to understand.
One cautionary point I ran into was that if your RAID array is larger than 2TB you will need to mount it after you have installed the OS. See the section on RAID for details.
***********************************************************************************
Debian Etch
I chose Linux as an operating system since it is designed for server applications and is Free and free. After some distribution shopping I arrived at Debian. I like Debian because it is extremely stable, well documented and I have been able to find good support since it is widely used. I am currently using the Debian Etch distribution.
Before you install Debian, you need to think about Samba. I know this sounds out of order, but if you have a Windows machine, the only way to make network access seamless is to have common usernames and passwords for the Windows login, the Debian login and the Samba login. For details take a listen to Linux Reality episode 26.
Installing Debian is quick and simple. I simply downloaded the ISO image for the net install version from the Debian server, burned it to a CD, put it in the CD reader and rebooted the computer. My BIOS setting looks for a bootable CD in the CD reader first, so you may need to make that change if yours does not. It will ask some questions about language and time then install the basic packages. It will reboot from the hard drive and then ask you if you want to set up any pre-configured stuff. I selected the basic package and the GUI.
Debian automatically puts the installation CD in the list of places to look for installing packages. This is annoying and completely unnecessary. To get rid of this, you need to edit a file called /etc/apt/sources.list and comment out the line that refers to the CD. The resulting text file will look similar to this
************
#
# deb cdrom:[Debian GNU/Linux 4.0 r0 _Etch_ - Official i386 NETINST Binary-1 20070407-11:29]/ etch contrib main
#deb cdrom:[Debian GNU/Linux 4.0 r0 _Etch_ - Official i386 NETINST Binary-1 20070407-11:29]/ etch contrib main
deb http://ftp2.de.debian.org/debian/ etch main
deb-src http://ftp2.de.debian.org/debian/ etch main
deb http://security.debian.org/ etch/updates main contrib
deb-src http://security.debian.org/ etch/updates main contrib
**********
I highly recommend setting the IP static. I have static IP addresses for everything on my network. You can do this through the Desktop|Administration|Network menu in Gnome or you can edit /etc/network/interfaces using your favorite text editor.
Then you will want to update everything that installed from the CD
#apt-get update
#apt-get upgrade
This is a list of "useful packages" I found on the internet. I think it was on howtoforge.net, but I can't remember exactly. I had trouble compiling a program the first time I installed Debian, but after I installed all this I had no trouble.
apt-get install binutils cpp flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libdb4.3-dev libpcre3 libpopt-dev linux-kernel-headers lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++
I started this project with Debian Etch as described below, but ended up switching to openSUSE 10.3 for support of my RAID card. When Debian Lenny become the stable release, I might look at switching back to Debian since I like their command line functionality better. Yast is nice, but sometimes I am not sure exactly what it is doing and it is a bit slower than Debian once you get used to the command line. OpenSUSE 10.3 was my second choice primarily because it used a Linux kernel >2.6.19, and secondly because it was mentioned in most of the RAID hardware literature.
Installing SUSE was easy. I downloaded the DVD ISO image from www.opensuse.org and burned it to a DVD. When I started the installation, the RAID card and drives were installed and a volume had already been created. It was pretty easy during installation to specify that the file structure be mounted on the OS drive with the exception of /home, which was mounted on the RAID array. The menus were all intuitive and easy to understand.
One cautionary point I ran into was that if your RAID array is larger than 2TB you will need to mount it after you have installed the OS. See the section on RAID for details.
***********************************************************************************
Debian Etch
I chose Linux as an operating system since it is designed for server applications and is Free and free. After some distribution shopping I arrived at Debian. I like Debian because it is extremely stable, well documented and I have been able to find good support since it is widely used. I am currently using the Debian Etch distribution.
Before you install Debian, you need to think about Samba. I know this sounds out of order, but if you have a Windows machine, the only way to make network access seamless is to have common usernames and passwords for the Windows login, the Debian login and the Samba login. For details take a listen to Linux Reality episode 26.
Installing Debian is quick and simple. I simply downloaded the ISO image for the net install version from the Debian server, burned it to a CD, put it in the CD reader and rebooted the computer. My BIOS setting looks for a bootable CD in the CD reader first, so you may need to make that change if yours does not. It will ask some questions about language and time then install the basic packages. It will reboot from the hard drive and then ask you if you want to set up any pre-configured stuff. I selected the basic package and the GUI.
Debian automatically puts the installation CD in the list of places to look for installing packages. This is annoying and completely unnecessary. To get rid of this, you need to edit a file called /etc/apt/sources.list and comment out the line that refers to the CD. The resulting text file will look similar to this
************
#
# deb cdrom:[Debian GNU/Linux 4.0 r0 _Etch_ - Official i386 NETINST Binary-1 20070407-11:29]/ etch contrib main
#deb cdrom:[Debian GNU/Linux 4.0 r0 _Etch_ - Official i386 NETINST Binary-1 20070407-11:29]/ etch contrib main
deb http://ftp2.de.debian.org/debian/ etch main
deb-src http://ftp2.de.debian.org/debian/ etch main
deb http://security.debian.org/ etch/updates main contrib
deb-src http://security.debian.org/ etch/updates main contrib
**********
I highly recommend setting the IP static. I have static IP addresses for everything on my network. You can do this through the Desktop|Administration|Network menu in Gnome or you can edit /etc/network/interfaces using your favorite text editor.
Then you will want to update everything that installed from the CD
#apt-get update
#apt-get upgrade
This is a list of "useful packages" I found on the internet. I think it was on howtoforge.net, but I can't remember exactly. I had trouble compiling a program the first time I installed Debian, but after I installed all this I had no trouble.
apt-get install binutils cpp flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libdb4.3-dev libpcre3 libpopt-dev linux-kernel-headers lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++
Wednesday, January 2, 2008
Computer
I am currently using an ACER computer with a dual core AMD 64 bit processor and 1 GB of RAM. This is overkill for my server but it was cheap (~$400).
The main other consideration is hard drive bays. Since I am going to have a RAID 5 array plus a separate hard drive for the OS, I need space for 5 3.5" hard drives. The specs on this computer listed 6 3.5" drive bays, but it turns out that two of them are really for floppy drives. A hard disk will fit, but it is pretty tight. Instead I bought some brackets that converted one of the 5.25" bays to a 3.5" bay.
Unless the motherboard of your computer comes with RAID built in, you will need to buy a RAID card. I highly recommend 3ware cards because drivers for their cards are part of the Linux kernel starting with 2.6.19. Most RAID cards will fit into some kind of PCI slot. I am using a PCIe 4x RAID card which ensures that the bus will never be the limiting factor for data transfer. Make sure the motherboard has the slots you need. My motherboard has a 1x and a 16x slot. Bus rates are listed below. Wikipedia has some handy references for device bandwidths
SATA II 3000 Mbps
PCIe 4x 8000 Mbps
PCI 32 bit 33 MHz 1067 Mbps
Gigabit Ethernet 1000 Mbps
So you can see it is nice to have the PCIe interface, but it really bottlenecks at the Ethernet adapter unless you are operating the computer locally.
The main other consideration is hard drive bays. Since I am going to have a RAID 5 array plus a separate hard drive for the OS, I need space for 5 3.5" hard drives. The specs on this computer listed 6 3.5" drive bays, but it turns out that two of them are really for floppy drives. A hard disk will fit, but it is pretty tight. Instead I bought some brackets that converted one of the 5.25" bays to a 3.5" bay.
Unless the motherboard of your computer comes with RAID built in, you will need to buy a RAID card. I highly recommend 3ware cards because drivers for their cards are part of the Linux kernel starting with 2.6.19. Most RAID cards will fit into some kind of PCI slot. I am using a PCIe 4x RAID card which ensures that the bus will never be the limiting factor for data transfer. Make sure the motherboard has the slots you need. My motherboard has a 1x and a 16x slot. Bus rates are listed below. Wikipedia has some handy references for device bandwidths
SATA II 3000 Mbps
PCIe 4x 8000 Mbps
PCI 32 bit 33 MHz 1067 Mbps
Gigabit Ethernet 1000 Mbps
So you can see it is nice to have the PCIe interface, but it really bottlenecks at the Ethernet adapter unless you are operating the computer locally.
Overview
I set out to create a home server having never used Linux before. This is therefore a summary from a newbie perspective. There is of course much more detailed information for everything I discuss here. This discussion is limited only to what I actually use. It is also intended to be my notes to rebuild a server if anything catastrophic happens.
It was not easy getting here, especially the RAID part, so I wanted to share what I have learned in this process to help others who are trying to do the same thing. After all it was only because of people freely sharing information on podcasts (www.linuxreality.com) message boards and tutorials that I was able to do this. I hope this helps someone.
The first step is to decide what you want the server to do. This will affect the hardware you buy and the operating system you select. I wanted my server to primarily be a file server, but a complete list of functions follows:
file server (samba)
web server (apache)
RAID array (3ware)
music server (tangerine)
media server (twonkymedia)
IMAP mail server (getmail and dovecot)
DNS and DHCP server (dnsmasq)
Once you have decided on functions you will need a computer and an operating system.
When it is done I want it to be a headless machine (no keyboard, mouse or monitor) so I have it set up with SSH and VNC. During the setup process you will want direct access, so it is also handy to have a KVM switch.
Once you have a server it makes backup easier because all your data is stored in one place, but it makes backup critical because all your data is stored in one place. I have implemented a 3 layer backup system that minimizes my risk of data loss.
Logging data can be useful. This is a topic I have yet to explore.
System automation such as software updates and periodic rebooting.
If you are looking for an excellent overview of Linux and Linux servers in general, you should check out Linux Reality podcast. That is where I got my start. I will link to specific episodes as I go.
It was not easy getting here, especially the RAID part, so I wanted to share what I have learned in this process to help others who are trying to do the same thing. After all it was only because of people freely sharing information on podcasts (www.linuxreality.com) message boards and tutorials that I was able to do this. I hope this helps someone.
The first step is to decide what you want the server to do. This will affect the hardware you buy and the operating system you select. I wanted my server to primarily be a file server, but a complete list of functions follows:
file server (samba)
web server (apache)
RAID array (3ware)
music server (tangerine)
media server (twonkymedia)
IMAP mail server (getmail and dovecot)
DNS and DHCP server (dnsmasq)
Once you have decided on functions you will need a computer and an operating system.
When it is done I want it to be a headless machine (no keyboard, mouse or monitor) so I have it set up with SSH and VNC. During the setup process you will want direct access, so it is also handy to have a KVM switch.
Once you have a server it makes backup easier because all your data is stored in one place, but it makes backup critical because all your data is stored in one place. I have implemented a 3 layer backup system that minimizes my risk of data loss.
Logging data can be useful. This is a topic I have yet to explore.
System automation such as software updates and periodic rebooting.
If you are looking for an excellent overview of Linux and Linux servers in general, you should check out Linux Reality podcast. That is where I got my start. I will link to specific episodes as I go.
Subscribe to:
Posts (Atom)
Posts
