#!/bin/bash
# Shell Script To List All Top Hitting IP Address to your webserver.
# This may be useful to catch spammers and scrappers.
# ———————————————————————-
# This script is licensed under GNU GPL version 2.0 or above
# ———————————————————————-
# where to store final report?
DEST=/var/www/reports/ips

# domain name
DOM=$1

# log file location
LOGFILE=/var/logs/httpd/$DOM/access.log

# die if no domain name given
[ $# -eq 0 ] && exit 1

# make dir
[ ! -d $DEST ] && mkdir -p $DEST

# ok, go though log file and create report
if [ -f $LOGFILE ]
then
echo “Processing log for $DOM…”
awk ‘{ print $1}’ $LOGFILE | sort | uniq -c | sort -nr > $DEST/$DOM.txt
echo “Report written to $DEST/$DOM.txt”
fi

How do I run this script?

Simply run it as follows:
./script website.com
Sample output (1st coloum is counter and 2nd is IP address):

600 72.30.87.116
50 66.249.71.138
10 66.249.71.140
5 66.249.71.139
3 74.86.49.130

#!/bin/bash
# Shell script to clean web server cache stored at /var/www/cache/ directory.
# ————————————————————————-
# This script is licensed under GNU GPL version 2.0 or above
# ————————————————————————-

# Cache dir path
CROOT=”/var/www/cachelighttpd/”

#Deleting files older than 10 days
DAYS= 10

# Lighttpd user and group
LUSER=”lighttpd”
LGROUP=”lighttpd”

# start cleaning
find ${CROOT} -type f -mtime +${DAYS} | xargs -r /bin/rm

# if directory missing just recreate it
if [ ! -d $CROOT ]
then
mkdir -p $CROOT
chown ${LUSER}:${LGROUP} ${CROOT}
fi

To install it you must download the source code and compile by running these commands:

make mod_dnsblacklist.o

gcc -shared -o mod_dnsblacklist.so mod_dnsblacklist.o

/usr/bin/install -c mod_dnsblacklist.so /usr/local/lib/mod_dnsblacklist.so

The module accepts these directives:

dnsblacklist.method
Syntax:   dnsblacklist.method string
Supported: GET, POST, HEAD, OPTIONS, PUT and PROPFIND
Default:  POST

The HTTP method on which the module acts

dnsblacklist.host
Syntax:   dnsblacklist.host string
Default:  sbl-xbl.spamhaus.org

The address of the DNSBL used

dnsblacklist.message
Syntax:   dnsblacklist.message string
Default:  Your IP address is blacklisted!

Error message displayed to the blocked user

Once installed you will need to enable it editing the Lighttpd’s configuration (/etc/lighttpd/lighttpd.conf). Here’s an example:

server.modules = (
…..
“mod_dnsblacklist”,
……

Finally you must restart the server

/etc/init.d/lighttpd restart

The default configuration will protect you from attacks performed with the POST method such spam relay via web forms and on your blog. To extend the protection and preventing URL injection put this in the configuration of Lighttpd:

dnsblacklist.method "GET,POST"

In order to change the error message shown to blocked users, you can use the directive “dnsblacklist.message” in this way:

dnsblacklist.message “Your custom message”

…and now fly light

Install rrdtool

Type the following command if you are using CentOS / RHEL / Fedora Linux (enable EPEL repo):
# yum install rrdtool
If you are using Debian / Ubuntu Linux, enter:
# apt-get update && apt-get install rrdtool

Configure mod_rrdtool

You need to use /usr/bin/rrdtool binary file. Open the lighttpd configuration file, enter:
# vi /etc/lighttpd/lighttpd.conf
Add the following:

server.modules += ( “mod_rrdtool” )
### RRDTOOL Config
# path to the rrdtool binary
rrdtool.binary = “/usr/bin/rrdtool”
# rrd database file
rrdtool.db-name = “/home/lighttpd/rrd”

Save and close the file. Now gracefully reload lighttpd webserver:
# service lighttpd reload
/home/lighttpd/rrd is a filename of the rrd-database. Make sure that /home/lighttpd/rrd doesn’t exist before the first run, as lighttpd has to create the DB for you.

How Do I View Graphs?

You need to download and install cgi script. My cgi-bin directory is located at /home/lighttpd/cgi-bin/ (see how to configure cgi-bin CGI access under Lighttpd):

$ cd /home/lighttpd/cgi-bin/
$ wget http://redmine.lighttpd.net/attachments/download/793
$ chmod +x lightygraph.cgi

Open the web browser and type url:
http://your.domain.com/cgi-bin/lightygraph.cgi

Sample graphs:

Fig.01: Lighttpd WebServer Statistics For Last 4 Hours

Fig.02: Lighttpd WebServer Statistics – Daily Graphs

Step #1: Create a CGI cgi-bin Directory

First you need to create a cgi-bin directory for your domain. Assuming that your domain hosted at /home/lighttpd/theos.in/http (DocumentRoot), create cgi-bin as follows:
# mkdir -p /home/lighttpd/theos.in/cgi-bin

Step # 2: Load mod_cgi Module

Open lighttpd configuration file using a text editor such as vi:
# vi /etc/lighttpd/ligttpd.conf

Now append or modify text as follows so that support for mod_cgi get loaded:

server.modules += ( “mod_cgi” )

Find out your virtual server configuration and append the following:

$HTTP["url"] =~ “/cgi-bin/” {
cgi.assign = ( “.pl” => “/usr/bin/perl” )
}

Here is complete my config code:

$HTTP["host"] =~ “theos.in” {
server.document-root = “/home/lighttpd/theos.in/http”
accesslog.filename = “/var/log/lighttpd/theos.in/access.log”
$HTTP["url"] =~ “/cgi-bin/” {
cgi.assign = ( “.pl” => “/usr/bin/perl” )
}
}

Step # 3:Restart the Lighttpd

Restart lighttpd webserver, enter:
# /etc/init.d/lighttpd restart

Step # 4: Test It

Create a file/perl program in /home/lighttpd/theos.in/cgi-bin/sample.pl:

#!/usr/bin/perl
print “Content-Type: text/plain”, “\n\n“;
print “Hi there! This is a sample perl program!!!”, “\n“;

Save and execute the program (http://yourdomain.com/cgi-bin/sample.pl).