FastCGI Engine Contrib

Permits Foswiki to be executed with FastCGI?

Overview

FastCGI is a technology to deliver dynamic web content. It differs from CGI cause it remains persistent between requests, instead of CGI approach of a new forked process per request. This way there is a significant performance improvement, since all overhead related to create a new process, load the interpreter and compile the code is skipped.

Some FastCGI features:

• The number of persistent processes is configurable, independent of the web server. This leads to easier capacity planning/management.
• Processes can be run with a different user: more security.
• Processes can be run on another machines: easier load balancing.

Installation Instructions

This section is about how to configure FastCGIEngineContrib, considering many possible environments:

• Apache web server
  • Using only .htaccess (typically on host services)
  • With access to apache configuration files
    • Remote FastCGI processes
  • Using mod_fcgid or mod_fastcgi
• Lighttpd web server
  • Remote FastCGI processes
• nginx web server
  • remote FastCGI processes

Installation of the FCGI CPAN library

FastCGIEngineContrib uses a CPAN library called FCGI which is not normally distributed with Perl. Version of FCGI should be 0.67 or later

Generic CPAN references as well as the most popular distribution packages are listed below

• CPAN - http://search.cpan.org/dist/FCGI/ (generic CPAN installation perl -MCPAN -e 'install FCGI')
• RedHat/Centos RPM from Dag - http://dag.wieers.com/rpm/packages/perl-FCGI/
• Mandriva/Suse/Fedora/Redhat RPM from Dries - http://dries.ulyssis.org/ayo/packages/perl-FCGI/info.html
• Debian - http://packages.debian.org/search?searchon=names&keywords=libfcgi-perl
• Ubuntu - http://packages.ubuntu.com/search?searchon=names&keywords=libfcgi-perl

Apache

Apache Module

There are two options that basicly do the same thing

• mod_fastcgi which is the oldest implementation. It is released under a custom non-free license but it is free of charge.
• mod_fcgid which is the newer implementation released under the GPL license and now part of the Apache Foundation.

It is difficult to recommend one or the other. In some cases one will work and the other will not. Most positive reports have been with mod_fcgidbut both have been tested and work with standard out-of-the-box Apache and Foswiki setups. mod_fcgid is easier to find as package because of its GPL license so it may be the best place to start. Fedora has mod_fcgid on the installation DVD now.

Below are some resources for the most common Linux distributions. The actual versions of the latest packages may have changed since this documentation was written.

mod_fcgid resources

• Sources - http://httpd.apache.org/mod_fcgid/
• RedHat/Centos EL5 64 bit - http://centos.karan.org/el5/extras/testing/x86_64/RPMS/mod_fcgid-2.2-4.el5.kb.x86_64.rpm
• RedHat/Centos EL5 32 bit - http://centos.karan.org/el5/extras/testing/i386/RPMS/mod_fcgid-2.2-4.el5.kb.i386.rpm
• Debian - http://packages.debian.org/search?searchon=names&keywords=libapache2-mod-fcgid

mod_fastcgi resources

• RedHat/Centos - http://rugmonster.org/2009/03/building-mod_fastcgi-on-rhel5/ and http://www.nagare.org/trac/blog/apache_mod_fastcgi
• Debian - http://packages.debian.org/unstable/httpd/libapache2-mod-fastcgi

Apache Configuration

It is strongly recommended that users work from Foswiki:Support.ApacheConfigGenerator to create initial Apache configurations. This config generator is comprehensive and well tested; the examples below may be inadequate for your needs.

Using only .htaccess

Using only .htaccess file it's not possible to control the number of FastCGI processes, nor the user it'll be run with. We assume the webserver is configured to recognize files with .fcgi extension to be FastCGI scripts. We also need mod_rewrite or mod_actions enabled.

Examples

Directory paths are relative to Foswiki root. Usage of mod_fcgid is assumed. If you're using mod_fastcgi, replace fcgid-script with fastcgi-script. mod_fcgid is preffered over mod_fastcgi. The latter project receives very few updates and has some known issues, such as Foswiki:Support.Faq18, whereas mod_fcgid is a default module that is shipped with Apache httpd.

Using mod_rewrite:

# bin/.htaccess file

Options +ExecCGI

RewriteEngine On

# configure script is a special case, it must run as a CGI script:
RewriteCond %{REQUEST_URI} !/configure 

# avoid loops:
RewriteCond %{REQUEST_URI} !/foswiki.fcgi

RewriteRule ^(.*) /foswiki/bin/foswiki.fcgi/$1


    SetHandler cgi-script

Using mod_actions:

# bin/.htaccess file

Options +ExecCGI

Action foswiki-fcgid /foswiki/bin/foswiki.fcgi
SetHandler foswiki-fcgid


    SetHandler cgi-script



   SetHandler fcgid-script

Using apache .conf

Direct access to Apache configuration files open more possibilities than with .htaccess, such as:

• Configurable number of FastCGI processes
• With mod_fastcgi:
  • Use remote FastCGI processes to service requests
    • This might be possible under mod_fcgid using a custom wrapper script with the FcgidWrapper directive and the virtual flag
  • Control the user the FastCGI processes run under

Using mod_fastcgi

Consider using the newer and better-maintained mod_fcgid which is shipped standard with Apache httpd.

We can use static or dynamic servers: static servers are initialized with Apache itself and exists as long Apache is running. Dynamic servers are loaded on demand and killed if the aren't needed anymore.

See mod_fastcgi documentation for more options.

# Simple and traditional example.
Alias /foswiki/bin/configure /var/www/foswiki/bin/configure
Alias /foswiki/bin   /var/www/foswiki/bin/foswiki.fcgi

# Commenting the next line makes foswiki to be a dynamic server, loaded on demand
FastCgiServer /var/www/foswiki/bin/foswiki.fcgi -processes 3


   Options +ExecCGI
    
        SetHandler cgi-script
    
    
        SetHandler fastcgi-script
    

Refer to tuning section below for a little discussion about the number of FastCGI processes.

# External server: could be running at another machine and/or a different user from the webserver

Alias /foswiki/bin/configure /var/www/foswiki/bin/configure
Alias /foswiki/bin   /var/www/foswiki/bin/foswiki.fcgi

# Running an external server on the same machine:
FastCgiExternalServer /var/www/foswiki/bin/foswiki.fcgi -socket /path/to/foswiki.sock

# Or at another machine:
FastCgiExternalServer /var/www/foswiki/bin/foswiki.fcgi -host example.com:8080


    Options +ExecCGI
    
        SetHandler cgi-script
    
    
        SetHandler fastcgi-script
    

When you're using external servers you must run the FastCGI processes manually:

$ cd /var/www/foswiki/bin
# To start a pool of processes, listening to a local UNIX socket:
$ ./foswiki.fcgi --listen /path/to/foswiki.sock --nproc 3 --pidfile /path/to/pidfile.pid --daemon
# Or listening to a local :port address:
$ ./foswiki.fcgi --listen :port --nproc 3 --pidfile /path/to/pidfile.pid --daemon

Run ./foswiki.fcgi --help for details on the options.

Using mod_fcgid

Comparison to mod_fastcgi:

• Compatible API from Foswiki's point of view, drop-in replacement
• Maintained as a part of the standard Apache httpd distribution
• Does not itself provide a mechanism to run the FastCGI processes as a different user (possible through other means)
• mod_fcgid documentation suggests that only dynamic servers are supported
• No (easy) way to use FastCGI processes running on another machine; but it might be possible with a custom wrapper script and the FcgidWrapper directive with the virtual flag

# Simple and traditional example.
Alias /foswiki/bin/configure /var/www/foswiki/bin/configure
Alias /foswiki/bin   /var/www/foswiki/bin/foswiki.fcgi


   Options +ExecCGI
    
        SetHandler cgi-script
    
    
        SetHandler fcgid-script
    

ApacheLogin

If you use ApacheLogin instead of TemplateLogin you'll need to add something like the following directives:

   AuthType Basic
   AuthName "Foswiki login realm"
   AuthUserFile "/var/www/foswiki/data/.htpasswd"
   Require valid-user

Lighttpd

# Example with FastCGI processes launched by the webserver
$HTTP["url"] =~ "^/foswiki/bin/configure" {
    alias.url += ( "/foswiki/bin" => "/var/www/foswiki/bin" )
    cgi.assign = ( "" => "" )
}

$HTTP["url"] =~ "^/foswiki/bin/" {
    alias.url += ( "/foswiki/bin" => "/var/www/foswiki/bin/foswiki.fcgi" )
    fastcgi.server = ( ".fcgi" => (
            (
                "socket"    => "/var/www/foswiki/working/tmp/foswiki.sock",
                "bin-path"  => "/var/www/foswiki/bin/foswiki.fcgi",
                "max-procs" => 3
            ),
        )
    )
}

# Example with external FastCGI processes (running on the same host, with another user or at a remote machine)
$HTTP["url"] =~ "^/foswiki/bin/configure" {
    alias.url += ( "/foswiki/bin" => "/var/www/foswiki/bin" )
    cgi.assign = ( "" => "" )
}

$HTTP["url"] =~ "^/foswiki/bin/" {
    alias.url += ( "/foswiki/bin" => "/var/www/foswiki/bin/foswiki.fcgi" )
    fastcgi.server = ( ".fcgi" => (
            (
                "host"    => "example.com",
                "port"    => "8080",
            ),
        )
    )
}

Nginx

In contrast to Apache or Lighttpd Nginx does not control the life time of the foswiki.fcgi backend process. Instead you will have to start it yourself using the system's init process. The FCGI::ProcManager class will then take care of (re-)spawning enough child processes as required.

First, let's configure nginx to contact a foswiki.fcgi process on some socket on the localhost:

server {
...
  root /var/www/foswiki;

  location = / {
    try_files $uri @foswiki;
  }

  location ~ ^/(?:/foswiki/bin/)?([A-Z_].*)$ {
    rewrite ^/(.*)$ /foswiki/bin/view/$1;
  }

  location ~ ^/pub/(System|Applications|images|cache)/ {
    expires 8h;
    gzip_static on;
  }

  location /pub {
    rewrite ^/pub/(.*)$ /foswiki/bin/viewfile/$1;
  }

  location /foswiki/bin {
    try_files $uri @foswiki;
  }

  location @foswiki {
     gzip off;

     fastcgi_pass 127.0.0.1:9000; 

     fastcgi_split_path_info ^/foswiki/bin/(.+?)(/.*)$;
     fastcgi_param SCRIPT_FILENAME /var/www/foswiki/bin/foswiki.fcgi;
     fastcgi_param PATH_INFO $fastcgi_path_info;

     include fastcgi_params;
  }

  location ~ (^/lib|^/data|^/locale|^/templates|^/tools|^/work) {
     deny all;
  }

...
}

Next, to integrate the foswiki.fgi process into the system's init process use the two helper scripts in the tools directory:

• foswiki.init-script: copy this to /etc/init.d/foswiki; make the file executable using chmod +x /etc/init.d/foswiki
• foswiki.defaults: copy this to /etc/defaults/foswiki and make appropriate adjustmenst; make sure the process uses the same socket as configured in nginx (see above, defaults to 127.0.0.1:9000)

You should now be able to control the backend processes using service foswiki start/stop/reload/restart/status.

Finally, add the service to the runlevels using update-rc.d foswiki defaults to make sure the service is started on system startup time.

Tuning

Except from Apache using only .htaccess file, it's possible to adjust the number of FastCGI processes. There is no magic number: it depends on some variables, like the hardware resources and access load. If you set this number too low, users may experience high latencies and you'll not use all hardware potential, on the other hand if this setting is adjusted too high then the server can be forced to use swap, what degrades performance a lot.

Dynamic servers are more useful when Foswiki access load on the server is low and/or it's used for something in addition to Foswiki. Under high loads, static servers can deliver better performance.

Known Issues

This is a persistent engine, so you need to restart the web server after some configuration update is performed. However, there is an auto-reload mechanism that apply changes without a web server restart. Unfortunately, there is a delay: after the update, each process will still serve one more request before reloading itself (e.g. if you're using 3 processes, the next 3 requests after the update will not be affected. The update will take effect on the requests made after the initial 3). This reloading mechanism works only on operating systems that have the exec(2) system call, like Linux and other POSIX compliant systems. All examples above have an exception to configure script. This script needs to run as a plain CGI script. There are some legacy extensions (not updated to Foswiki:Development/FoswikiStandAlone design) that adds scripts to the bin/ directory. You need to add exceptions for these scripts as well. FastCGI support on IIS 6.0 (and maybe other versions) is broken with respect to the STDERR stream. This may cause problems.

Info