Creation of a Server Environment
From I571/ChE531 2007 Class Wiki
[edit] Hardware & Software
The following is recommended as a basic server setup for a small scientific software development group:
- A desktop PC (Min Celeron 2GHz, 512MB Ram, 80GB Hard Disk) connected to internet or intranet
- Red Hat, Suse or Ubuntu Linux
- Apache Web Server (usually supplied with Linux, possibly extended with WebDAV and SSL)
- Perl, MySQL and PHP (usually supplied with Linux]
- Subversion (or CVS) for source code management (usually supplied with Linux)
- MediaWiki for group collaboration
- Bugzilla for bug tracking and quality control
[edit] Accessing Cheminfo (or another Unix server)
- On the PC, I like Putty
- On the Mac, just open a shell
- Connect to cheminfo with
ssh yourname@cheminfo.informatics.indiana.edu
[edit] Installing Apache (in your own directory)
If you really have to try this on cheminfo, please remember to use a different number than 8080 (e.g. 8081), and do an apachectl stop when you are done
Download relevant binary from http://mirror.cc.columbia.edu/pub/software/apache/httpd/binaries/linux/ e.g.
wget http://mirror.cc.columbia.edu/pub/software/apache/httpd/binaries/linux/apache_1.3.33-i686-whatever-linux22.tar.gz
Uncompress the files and go into the new directory
tar xvzf apache_1.3.33-i686-whatever-linux22.tar.gz cd apache_1.3.33
Install the binary as follows (set the directory you wish to use, default /usr/local/apache)
./install-bindist.sh /home/djwild/i573/apache
Go into the configuration directory
cd /home/djwild/i573/apache/conf
Edit the configuration file, e.g.
emacs httpd.conf
Change variables as necessary in Section 2. For our version we'll just change the port:
Port 8080
Start apache
/home/djwild/i573/apache/bin/apachectl start
Try it out! - http://cheminfo.informatics.indiana.edu:8080 To stop the server, type
/home/djwild/i573/apache/bin/apachectl stop
[edit] Setting up a WebDAV server
A WebDAV server allows you to have a read/writable directory for file sharing that is accessible through the HTTP protocol. For more information, see the WebDAV homepage. The basics are:
Ensure that the mod_dav module is loaded into Apache. This can be done by looking in the /etc/sysconfig/apache2 file (SuSe), or looking for LoadModule commands in httpd.conf or one of its derivative files. If you need to install mod_dav, you can do it as a dynamic module without re-compiling apache by downloading the tar file from http://www.webdav.org/mod_dav/. Here is how we will do it:
cd /home/djwild/i573 wget http://www.webdav.org/mod_dav/mod_dav-1.0.3-1.3.6.tar.gz tar xvzf mod_dav-1.0.3-1.3.6.tar.gz cd mod_dav-1.0.3-1.3.6
We now need to set the Perl directory (/usr/bin/perl) in the apxs program:
vi /home/djwild/i573/apache/bin/apxs
Now we can configure and install the DAV module
./configure --with-apxs=/home/djwild/i573/apache/bin/apxs make make install
Now we just need to create a directory for the DAV lock file and for the DAV directory:
mkdir /home/djwild/i573/apache/var mkdir /home/djwild/i573/apache/htdocs/dav
We can now tell httpd.conf where the lock file is and create a directory entry for the area you want to make operate as a DAV, e.g.:
DAVLockDB "/home/djwild/i573/apache/var" <Directory "/home/djwild/i573/apache/htdocs/dav"> Dav On Options +Indexes IndexOptions FancyIndexing IconsAreLinks </Directory>
You might wish to add authentication
<Directory "/home/djwild/i573/apache/htdocs/dav">
Dav On
Options +Indexes
IndexOptions FancyIndexing IconsAreLinks
AllowOverride AuthConfig
AuthName "I573 Dav Server"
Authtype Basic
AuthUserFile "/home/djwild/i573/apache/htpasswd"
Order allow,deny
Satisfy any
Require user testdav
</Directory>
In which case you will need to make a password file:
cd /home/djwild/i573/apache htpasswd -c htpasswd testdav
Note that to be secure, the DAV server should be on a secure SSL website (see below) so that passwords are encrypted.
To access the DAV server from a PC, you can either add the URL as a new drive, or use Connect To... On a Mac, in Finder go to Go -> Connect To Server and type the URL
Note for Microsoft Windows users: Windows XP doesn't by default permit connection to WebDAV servers with SSL security. If you try to connect to a secure server and get a message about "folder does not appear to be valid" you might need to install a patch.
[edit] Setting up a Secure (SSL) Webserver
A secure web server is different from a regular one in that it:
- Runs on port 443 instead of port 80
- Is accessed through https:// rather than http://
- Securely encrypts all communication
For instructions on how to set up a server, see Van's Apache SSL mini how-to (good for setting up certificates), Apache SSL FAQ, Sam's publishing Howto (good for httpd.conf setup). We're not going to do this for the course.
Here are example command lines to create certificates. All must be executed as root. It is suggested you create the certificates somewhere well away from the web server.
Generate certificate signing authority
openssl req -new -x509 -days 3650 -key fred-ca.key -out fred-ca.crt
Generate key and certificate for the website, using signing authority
openssl genrsa -des3 -out fredandwilma.key 1024 openssl req -new -key fredandwilma.key -out fredandwilma.csr openssl x509 -req -in fredandwilma.csr -out fredandwilma.crt -sha1 -CA fredandwilma.crt -CAkey fredandwilma.key -CAcreateserial -days 3650
Here is an example Virtual Hosts section of the httpd.conf file that sets up a secure server on an existing unsecure server, using a key and certificate that have already been generated:
<VirtualHost _default_:443>
DocumentRoot "/srv/www/securedocs"
ServerName www.fredandwilma.com
SSLEngine on
SSLCertificateFile /etc/apache2/ssl.crt/fredandwilma.crt
SSLCertificateKeyFile /etc/apache2/ssl.key/fredandwilma.key
</VirtualHost>
Some notes and gotchas:
- SSL doesn't password protects, it just provides a secure encrypted environment where passwords, etc can be sent safely. It's fine to use e.g. Apache Authtype Basic security for passwords
- Under "Common Name" when generating the certificate, you must give the fully qualified domain name of the web site that you're generating the certificate for
- You can run both HTTP (80) and HTTPS (443) in the same instantiation of Apache
- Some implementations (e.g. SuSe) automatically turn on SSLEngine when you activate mod_ssl, meaning that it wants a certificate for the whole web site (including the 80) not just the Virtual Server that is running the HTTPS. Search for the line and remove it
- When starting apache, you need to specifically start it in SSL mode (e.g. apachectl start ssl)
- You cannot have multiple named virtual servers for 443 - you can only have one per IP (for a technical reason)
- You might explicitly have to tell the server to listen on port 443 as well as 80 (httpd.conf)
- Make sure your firewall (software and hardware) routes port 443 as well as 80
- When Apache starts up, you will have to enter the SSL password
[edit] Installing PHP
Downloading
wget http://us2.php.net/get/php-5.2.0.tar.gz/from/this/mirror php-5.2.0.tar.gz tar xvzf php-5.3.0.tar.gz
Configure and make
cd php-5.2.0 ./configure --prefix=/home/djwild/i573/php --with-apxs=/home/djwild/i573/apache/bin/apxs --with-mysql make make install
Edit httpd.conf to understand .php extensions and to respond to php index files
AddType application/x-httpd-php .php AddType application/x-httpd-php-source .phps ... DirectoryIndex index.html index.php main.php
[edit] Setting up MediaWiki
- Download mediawiki tar file, e.g.
wget http://umn.dl.sourceforge.net/sourceforge/wikipedia/mediawiki-1.9.0.tar.gz
- Extract tar file in directory where web pages are held (htdocs):
tar -xvzf mediawiki-1.9.0.tar.gz
- Rename mediawiki-1.9.0 directory to whatever you want to call the wiki
- Go into the directory, and make the config directory writable:
chmod a+w config
- Go to http://yourserver/wikiname and follow setup instructions
- Remember to move the LocalSettings.php file from the config directory up one level and remove the config directory
- Go to http://yourserver/wikiname and it should now work
[edit] Using Subversion on the local machine
Why use Subversion?
- Most current CVS features.
- Directories, renames, and file meta-data are versioned.
- Commits are truly atomic.
- Apache network server option, with WebDAV/DeltaV protocol.
- Standalone server option.
- Branching and tagging are cheap (constant time) operations
- Natively client/server, layered library design
- Client/server protocol sends diffs in both directions
- Costs are proportional to change size, not data size
- Choice of database or plain-file repository implementations
- Versioning of symbolic links
- Efficient handling of binary files
- Parseable output
- Localized messages
If you still want to use CVS, see Setting up CVS.
We will be using the global subversion on cheminfo, not one in our scratch area.
Create a (central) subversion repository
cd /home/djwild/i573/apache/htdocs mkdir svn svnadmin create /home/djwild/i573/apache/htdocs/svn
Create a local directory for files
cd /home/djwild/i573
mkdir svn
cd svn
mkdir branches
mkdir tags
mkdir trunk
cd trunk
cat > test.c
#include <stdio.h>
main()
{
printf("Hello World");
}
Do an initial import
svn import /home/djwild/i573/svn file:///home/djwild/i573/apache/htdocs/svn -m "initial import"
Make a local copy
cd /home/djwild/i573 mkdir svn2 cd svn2 svn checkout file:///home/djwild/i573/apache/htdocs/svn
Make changes, then use the following commands
svn diff test.c svn commit test.c -m message svn update
[edit] Using Subversion through WebDAV
* Coming soon!
[edit] Setting up Bugzilla
- Ensure Apache, MySQL and PHP are installed correctly
- Add a user to MySQL my typing the following, entering the password you wish to use instead of $db_pass (MySQL 4.0 or greater):
$ mysql
mysql> GRANT SELECT, INSERT,
UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES,
CREATE TEMPORARY TABLES, DROP, REFERENCES ON bugs.*
TO bugs@localhost IDENTIFIED BY '$db_pass';
mysql> FLUSH PRIVILEGES;
or for other versions:
$ mysql
mysql> GRANT SELECT, INSERT,
UPDATE, DELETE, INDEX, ALTER, CREATE, DROP,
REFERENCES ON bugs.* TO bugs@localhost IDENTIFIED BY
'$db_pass';
mysql> FLUSH PRIVILEGES;
- Download Bugzilla tar file (e.g. bugzilla-2.18.tar.gz)
- Extract tar file in directory where web pages are held (htdocs):
tar -xvzf bugzilla-2.18.tar.gz
- Rename bugzilla-2.18.tar.gz to whatever you want to call it (e.g. bugzilla)
- Make sure all the files in the directory are owned by the user that Apache runs under, e.g.:
chown -R http bugzilla chgrp -R http bugzilla
- Go into directory and type
./checksetup.pl
and follow any instructions, including setting up an administrator account.
- Edit the file localconfig to put the MySQL password you selected into the $db_pass field. You also might need to change the $webservergroup value
- Rerun ./checksetup.pl
- Add the following in the <Directory> section of the apache configuration file (/etc/httpd/conf/httpd.conf), changing as necessary for your setup:
<Directory "/usr/local/apache2/htdocs/bugzilla"> Options +ExecCGI +FollowSymLinks AllowOverride Limit </Directory>
- Add index.cgi to the list in the DirectoryIndex field in your apache configuration file
- Uncomment the following line in the apache configuration file, and restart the web server
AddHandler cgi-script .cgi
- Go into Bugzilla directory and type
./testserver.pl http://yourserver/bugzilla
(or whatever) to check configuration
- Start Bugzilla by going to http://yourserver/bugzilla/index.cgi. Log with the administrator email and password you created above, and then click on Edit Parameters at the bottom for more setup.