ClueBringer for POSTFIX Setup

May 5th, 2009

Blue: ClueBringer Install File
RED: Change

ClueBringer POLICYD is an addon for postfix that limits the amount of mail shared web hosting users can send per hour.

Integrating ClueBringer for PostFix has not been the easiest of tasks especially when a complicated setup exists, Here i will try to make sense of the entire program, POLICYD cluebringer is written in Phython that i know very little about, i will try to make sense of the code with my C knowledge.

To begin with, i will go about this in the same order the installation file requieres.

1. Setup MySQL (or whichever database you plan to use)

a. In database/ run…
$ for i in core.tsql access_control.tsql quotas.tsql amavis.tsql checkhelo.tsql checkspf.tsql greylisting.tsql
$ do
$ ./convert-tsql mysql $i
$ done > policyd.mysql

This will build policyd.mysql, be sure to ALWAYS load core.tsql first, you only really need the
schema for the modules you plan to use, no harm in adding all of them though.

b. Load policyd.mysql into MySQL
$ mysql -u root -p policyd < policyd.mysql

Done with the trivial creation of database and users, converting tsql files to a mysql file and importing it, and having a database ready for cluebringer.

2. Put cluebringer.conf in your /etc directory and adjust cluebringer.conf with your MySQL database details

Let us take a look at the cluebringer.conf file, Other thatn copeying it to the /etc/ directory i will also raise the log level to 4 for debugging

log_level=4

And point the cluebringer log file to /var/log/cbpolicyd.log

log_file=/var/log/cbpolicyd.log

I will also fix the database credentials to use MySQL, and create a file for the log that cluebringer policyd can write to

3. Copy the cbp/ directory into /usr/local/lib/policyd-2.0/
mkdir /usr/local/lib/policyd-2.0
cp -r cbp /usr/local/lib/policyd-2.0/

Taking a look inside, I found a bunch of interesting files .PM, looked inside, nothing to do there.

4. Copy cbpolicyd and cbpadmin into /usr/local/bin
cp cbpolicyd cbpadmin /usr/local/bin/

Those are 2 interesting PERL files, they start with

use lib(‘/usr/local/lib/policyd-2.0′,’/usr/lib/policyd-2.0′);

but there is no /usr/lib/policyd-2.0, No need to reach for the good old camel book yet, it turns out that adding this (Non existing location) to the include directory of PERL is not really a big deal.
5. Install the webui/* into your apache directory, check out includes/config.php and adjust the MySQL server details.

No rocket science there.

So, up to this minute, no change has been done to the system, You see everything we have done up to this minute is simply adding files and configuring them to wait for some other program to execute them, No change has been done to the system up to now.

The first change that will be done to the system is when we approach POSTFIX to add POLICYD to its list of consultants.

Although we will not be asking it to do it directly (We will ask amavisd-new to do it), we do need to add some rules to postfix, according to the installation file.

6. Setup Postfix to use cbpolicyd…
Add the following Postfix config…
check_policy_service inet:127.0.0.1:10031
in BOTH smtpd_recipient_restrictions and smtpd_end_of_data_restrictions.

In short, they want me to edit /etc/postfix/main.cf , If this is the first time you hear of this file, Or if you don’t know what the risk of being an open relay is, or if you don’t know what an open relay is, or if you know very little about email and the internet in general, and if you don’t know what SPF means, or if you don’t know how to write to this file, this tutorial is not yet for you, you need to start with a postfix book not here, You have been warned.
7. Copy your amavisd from /usr/sbin/amavisd and patch it with contrib/amavisd-new-2.5.3_policyd-200802070918.patch like this…
$ patch < contrib/amavisd-new-2.5.3_policyd-200802070918.patch
After this change your initscripts to start the patched amavisd.

AMAVISD-NEW, The patch that comes with cluebringer does the following to the amavisd-new file that is written in PERL.

———————————-

Ads 2 parts to the script

1- First it ads (The part in RED)

sub new { my($class,$conn,$msginfo) = @_; undef }
sub checks { my($self,$conn,$msginfo) = @_; undef }
sub before_send { my($self,$conn,$msginfo) = @_; undef }
sub after_send { my($self,$conn,$msginfo) = @_; undef }
sub mail_done { my($self,$conn,$msginfo) = @_; undef }
sub process_policy { my($self,$conn,$msginfo,$pbn) = @_; return $pbn }

2- Then it ads a part (In RED)

my($cnt_local) = 0; my($cnt_remote) = 0;
for my $r (@{$msginfo->per_recip_data}) {
my($recip) = $r->recip_addr;
my($is_local) = lookup(0,$recip, @{ca(‘local_domains_maps’)});
$is_local ? $cnt_local++ : $cnt_remote++;
$r->recip_is_local($is_local);
if (!$r->bypass_banned_checks) {
my($bypassed_b) = lookup(0,$recip, @{ca(‘bypass_banned_checks_maps’)});
$r->bypass_banned_checks($bypassed_b);
}
if (!$r->bypass_spam_checks) {
my($bypassed_s) = lookup(0,$recip, @{ca(‘bypass_spam_checks_maps’)});
$r->bypass_spam_checks($bypassed_s);
}
}

# Check if we need to do last minute policy changes
if (ref $custom_object) {
$which_section = “custom-process_policy”;
eval {
my $new_policy_bank = $custom_object->process_policy($conn,$msginfo,\%current_policy_bank); 1;
%current_policy_bank = %{ $new_policy_bank };
} or do {
my($eval_stat) = $@ ne ” ? $@ : “errno=$!”; chomp $eval_stat;
do_log(-1,”custom process_policy error: %s”, $eval_stat);
};
section_time($which_section);
}

———————————————————————————————–

The initscript that does the amavisd-new initialisations is /etc/init.d/amavisd
8. Copy amavisd-policyd.pm where you see fit, and add this configuration line to your amavisd.conf file…
include_config_files(‘/path/to/amavisd-policyd.pm’);

9. Edit the top of amavisd-policyd.pm and change …
use lib(‘/root/cluebringer-trunk’); # to point to the directory which contains cbp/

my $DB_dsn = “DBI:SQLite:dbname=/tmp/cluebringer.sqlite”; # to point to your MySQL DB

10. Fire everything up and browse to the web gui to configure

moving the blog

May 4th, 2009

So, This blog is moving to a much faster host !

Not exactly but here is how it is.

I have setup a server that is specialised in WordPress, Joomla, and other very popular PHP scripts

The morale (And what’s new) abou this is the speed, i will tell you about it in a minute

After making a server in over 2 days nonstop work, And after securing it from spammers, hackers, and other annoying people, i modified the WordPress system to use non-trivial software on the server.

Don’t get me wrong, WordPress is one fine software and so is Joomla, but they have been written to work on as many systems as possible, And not tailored to a certain system and it’s resources.

So, i will be selling Lightning Fast WordPress setup, With modified WordPress files to run on the system !

The system is expected to go live in 3 to 5 days, and will offer a free 1 month no credit card requiered trial, For those who opt to buy the service right away, the trial month is added to there account as a free month !

So, to top things off, A live chat person will actually set things up for the clients ! This is not just support, this is a managed blog !

The offering is a 5GB space, on an 8 Xeon Processor server (Two Quad Xeon Processors) with 8GBs of ram serving WordPress, this is nothing like our excellent PLESK offering, this is something you can find nowhere on the internet !

I will keep you updated on how things go as soon as we finish testing the server, Some very uncommon techniques are used to speed up your wordpress blog on this server, And for the same price as all our hosting level1 plans ($2.8/Mo), 5GB space and 100 email mailboxes, and a bunch of other nice stuff

Users on this server will not be restricted in any way, this is simply value added with no compromise for bloggers who want faster blogs (For both admin pannel and blog).

The only restriction to users who have not yet opted to use the server is email limit, this is because spammers usually hit on freee hosting, and we don’t want this server that we have spent so much time setting up blacklisted or sending spam to anyone, This restriction is removed as soon as the user is a paying customer !

Horde webmail : Content Encoding Error (content_encoding_error)

May 2nd, 2009

So, users were getting

Content Encoding Error (content_encoding_error)

Server response could not be decoded using encoding type returned by server.
This is typically caused by a Web Web site hosting presenting a content encoding header of one type, and then encoding the data differently.

For assistance, contact Customer Support.

And they did

Turned out horde comression for some encodings has issues, Solution was editing C:\Inetpub\vhosts\webmail\horde\config\conf.php, and change $conf['compress_pages'] to false (Turn off compression)

Everything works fine for now, If i intend to host any mail on a Windows server that is.

Changing GoDaddy secureserver PLESK server hostname

May 2nd, 2009

So, changing the hostname on My CentOS Virtual Machine keeps reverting back to ip-xxx-xxx-xxx-xxx.ip.secureserver.net, Not exactly what i want, especially when there is a 1,000 Emails mail quota for all godaddy VPS, i don’t really use this particular box for anything important, But i do need to have things right on every single box i run, With 10 Servers and 2 Virtual Dedicated servers, i really can use every peice of related information i can use.

So, the PLESK over virtuzzo has a license key, and i very much doubt that it is related to the hostname, but i will keep you posted once i finish the following steps

1- Adding the domain name (Sub domain as domain) to PLESK so that mail routs internally without going to godaddy once things change, So servername.powerdonkey.com domain gets added to PLESK.

Changing in the following places

/etc/hosts

/etc/sysconfig/network

/etc/network/interfaces
Then /etc/init.d/networking restart
/var/qmail/control/me
/etc/httpd/conf/httpd.include
hostname servername.powerdonkey.com
/etc/hostname

/etc/init.d/hostname.sh start

For testing

hostname
hostname -f

In the psa mysql database , in misc table (FullHostName, sso_relay, sso_server, sso_server_user_url), While the first updates from within CP, the other two do not !

Now, what about that… will it requier a full server reboot, or just

/etc/init.d/psa stopall
/etc/init.d/psa stop
/etc/init.d/psa start

auto updating WordPress and making it faster

May 2nd, 2009

Faster Auto Updating WordPress, Joomla, and other PHP scripts

So, we don’t think the popular PHP software packages are optimized at all (Knowing the developers had to make them cross platform compatible), We also think keeping them up to date is a hassle, and Auto installation scripts can mess up quite allot when they update, if you chose to modify your software, The auto updater can overwrite .

So, what has actually made it somewhere decent on the priority heap of my schedule is a modified version of this software that runs faster and auto updates, This should be added to the list of features in our standard hosting.

I will let you know how it goes, Very soon

catching spammers

May 1st, 2009

So, it is not only that i hate spammers, they also hurt us.

If a client we have is a spammer, the mail servers could get “Banned” at major email providers including Hotmail, Gmail, Yahoo, etc… and we can not afford legitimate client mail delivered to Spam folders or blocked all together

We do have measures to limit the amount of email sent per domain, But an in depth analysis of the mail is also important.

So at the office, i setup an UBUNTU linux machine and installed ISOQMAIL, ./configure make then make install

On the server, i made a job to copy the mail log (/usr/local/psa/var/log/maillog) to a password protected, web accessible directory, On the local machine i made a script to fetch the files into a directory known to isoqlog.

So now i need to make 2 files, one with the domains and one with the config, the file locations of the logs isoqlog should expect to find logs in and the place it should output the data to !

As for catching the PHP scripts that actually send the mail, here is a trick… PHP sends through sendmail, and we can actually have an intermediate script that verifies the mail, Limits it, or does anything else that is needed before actually sending the mail.

1- Add a wrapper script file (/var/qmail/bin/mailwrap), this can be a PHP script if you like to eecute at command line, A copy should go to the actual mail script, the other to wherever you want to log or control activity !

Plesk for Windows Auto Update

April 29th, 2009

Plesk for Windows does NOT auto update, the feature is entierly not implemented.

What you do it Download plesk for Windows from the Parallels website, and run the installer, I upgraded today from 9.0.1 to 9.2.1

Sounds like it went smoothly (Did not check yet), but the update from 8.6 to 9.0.1 went smooth that is for sure

As for this update, the setup file (About 600MB) can be extracted if you open it with 7ZIP

Setting up MySQL on Ubuntu

April 29th, 2009

Setting up MySQL to connect to from PLESK for example is very simple

apt-get install mysql-server

Then, you will need to allow mysql to listen to the outside world

Edit /etc/mysql/my.cnf

and comment out the line

bind-address = 127.0.0.1

then, you will need to allow the root user to connect from outside the server

On the command prompt

mysql -uroot –password=”aPassWord”

use mysql;
select host, user from user;

Now look for the root user with the hostname of the machine, assuming it came out as thishost

update user set host=’%’ where user=’root’ and host=’thishost’;

flush privileges;

quit

And you should be all set

If you don’t perform the above operations, PLESK will complain when you add the server to the list of database servers, You will probably see an error like

Error: Test connection to the database server has failed because of network problems:
Failed to connect to database: Can’t connect to MySQL server on ‘host.domain.com’ (10061)

IIS application pool from within PLESK

April 27th, 2009

The three options for this on a PLESK server running plesk 9.01 and IIS6 is as follows

Assignment and placement policy

We will choseto always have everyone in the shared application pool, CPU limit will come after some investigation

Linking Windows Plesk to external Database

April 27th, 2009

So, I setup Windows in a VM with that old version of windows i had on the shelf for some time. (Thanks XEN HVM), Installed .NET 3.5 SP1 and the KB959209 , Setup MSSQL Express 2008

Things that got setup

SQL SERVER REPLICATION
FULL TEXT SEARCH
REPORTING SERVICES
BUSINESS INTELLIGENCE DEV STUDIO
Management Tools
SQL Client Connectivity
ms sync framework

From the SQL SERVER CONFIGURATION MANAGER, i enabled TCP/IP from protocols for SQL express

Opened a hole in the firewall for both MSSQL\Binn\sqlservr.exe and \90\Shared\sqlbrowser.exe

and then from within Plesk i added the server

Server Host: mywin.server.com.\MSSQLExpress

Admin User: sa

Admin Pass: Same pass i entered in Mixed authentication for the sa user !

Done, Plesk thinks this is a great thing i have done and accepts the thing instantly

MS advises that i read Understanding Surface Area – In setup help Minimize SQL Server 2008 surface Area