HOWTO install Enkive 1.4 on Ubuntu 12.04 LTS

From Enkive Wiki

Revision as of 22:03, 5 March 2014 by Thomas (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

This HOWTO documents step-by-step how to install and configure Enkive 1.4 on Ubuntu 12.04 LTS (Precise Pangolin).

Before continuing, you should read the official, and generic, installation instructions for Enkive 1.4 at http://wiki.enkive.org/index.php/1.4_Installation_Instructions.

Assumptions: This will be a standalone server that only runs the Enkive email archiver. Any email sent to this machine's postfix (port 25) gets archived. In a production environment you will have to configure your existing mail server(s) to send a copy of each email that comes in to and goes out of your organization to this Enkive server. Configuring your existing mail server(s) to send duplicate emails to Enkive is not covered in this HOWTO.

The above assumes that the production mail server that will be sending copies of email to Enkive is on the same local network. It may be on a different subnet; for example, your mail server might have an IP address of 172.16.1.5 and the Enkive server an IP address of 10.1.1.20. Then the "mynetworks" line in main.cf might look like:

mynetworks = 127.0.0.0/8 172.16.1.0/16 10.1.1.0/24 [::ffff:127.0.0.0]/104 [::1]/128

which would accept mail from anyone on the 172.16.1.0 and 10.1.1.0 subnets.

When Enkive is finally up and running you can change the "mynetworks" line to restrict mail, allowing it from just a certain IP, for example.

After you have saved /etc/postfix/main.cf, reload postfix:

sudo /etc/init.d/postfix reload

Development environment

Some of the Enkive-related packages need to be compiled, so get a development environment installed. At minimum, the packages you'll need are:

gcc
g++
make
libcap-dev

sudo apt-get install gcc g++ make libcap-dev

Alternatively, installing the "build-essential" package will also get you most packages needed for compiling:

sudo apt-get install build-essential libcap-dev

Java

Unpack the tarball in /opt:

sudo tar zxvf jdk-7u51-linux-x64.tar.gz -C /opt/

Make a symbolic link:

cd /opt/
sudo ln -s jdk1.7.0_51/ java

So, per the official Enkive instructions, $JAVA_HOME is: /opt/jdk1.7.0_51 or /opt/java. We'll use /opt/java as the $JAVA_HOME from now on.

You may want to put /opt/java/bin in your path. There are various ways to do this. You can modify your .bashrc file, or even the /etc/environment file (which applies system-wide). Remember to log out and log back in for the changes to take effect.

Jsvc

You must use Jsvc version 1.0.12 (or later) with Enkive 1.4. (Version 1.0.15 is recommended.) Prior versions of Jsvc (1.0.11 and earlier) are not compatible with Enkive 1.4. Do not use the Jsvc from the Ubuntu 12.04 repository.

You can download previous versions of Jsvc at http://archive.apache.org/dist/commons/daemon/source/.

For Jsvc, we will compile with the java option and then copy the resulting binary to another directory so that it's in our path. Note that $JAVA_HOME is /opt/java:

tar zxvf commons-daemon-1.0.15-native-src.tar.gz
cd commons-daemon-1.0.15-native-src/
cd unix/
./configure --with-java=/opt/java
make
sudo cp jsvc /usr/local/bin/

Now "jsvc" is in your path. This is also where the enkive initscript expects to find the jsvc binary.

Swig

For swig:

sudo apt-get install swig zlib1g zlib1g-dev

Indri

For indri, we're going to install it in /opt/indri, as the official documentation suggests.

tar zxvf indri-5.6.tar.gz
cd indri-5.6/
./configure --prefix=/opt/indri-5.6 --enable-java --with-javahome=/opt/java
make
sudo make install

Create a symbolic link for /opt/indri:

cd /opt/
ln -s indri-5.6/ indri

MongoDB

There are several different ways to install MongoDB.

The easiest may be to use the custom MongoDB repository that the MongoDB maintainers offer. See the following link for how to install and configure:

http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/

(The MongoDB that is part of the Ubuntu 12.04 repository is slightly older, version 2.0.4. This will work, but we recommend using a 2.4.x MongoDB.)

If you just use the tarball provided on the downloads page -- mongodb-linux-x86_64-2.4.9.tgz -- keep in mind that it does not come with an initscript or upstart script.

An upstart script can be found in the Enkive tarball in the scripts/init/ubuntu/ subdirectory. Or you can use the SysV initscript below.

If you're using the mongodb-linux-x86_64-2.4.9.tgz tarball, unpack it in /opt. Create a symbolic link for it. Create a directory within it (ie, data/db) for the database files.

sudo tar zxvf mongodb-linux-x86_64-2.4.9.tgz -C /opt/
cd /opt/
sudo ln -s mongodb-linux-x86_64-2.4.9/ mongodb
cd mongodb/
sudo mkdir -p data/db

Create the initscript for mongodb, and put the following in /etc/init.d/mongodb:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          mongodb
# Required-Start:    $remote_fs $syslog $network
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: MongoDB initscript
# Description:       This file should be used to construct scripts to be
#                    placed in /etc/init.d.
### END INIT INFO

# Author: Foo Bar <foobar@baz.org>
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.

# Do NOT "set -e"

# PATH should only include /usr/* if it runs after the mountnfs.sh script
#PATH=/sbin:/usr/sbin:/bin:/usr/bin
PATH=/opt/mongodb/bin:/sbin:/usr/sbin:/bin:/usr/bin
DESC="MongoDB"
NAME=mongod
DAEMON=/opt/mongodb/bin/$NAME
#DAEMON_ARGS="--options args"
DAEMON_ARGS="--fork --dbpath /opt/mongodb/data/db --logpath /var/log/mongodb.log --logappend"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/mongodb

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
	# Return
	#   0 if daemon has been started
	#   1 if daemon was already running
	#   2 if daemon could not be started
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
		|| return 1
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
		$DAEMON_ARGS 2>&1 >>/var/log/mongodb.log \
		|| return 2
	# Add code here, if necessary, that waits for the process to be ready
	# to handle requests from services started subsequently which depend
	# on this one.  As a last resort, sleep for some time.
}

#
# Function that stops the daemon/service
#
do_stop()
{
	# Return
	#   0 if daemon has been stopped
	#   1 if daemon was already stopped
	#   2 if daemon could not be stopped
	#   other if a failure occurred
	start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
	RETVAL="$?"
	[ "$RETVAL" = 2 ] && return 2
	# Wait for children to finish too if this is a daemon that forks
	# and if the daemon is only ever run from this initscript.
	# If the above conditions are not satisfied then add some other code
	# that waits for the process to drop all resources that could be
	# needed by services started subsequently.  A last resort is to
	# sleep for some time.
	start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
	[ "$?" = 2 ] && return 2
	# Many daemons don't delete their pidfiles when they exit.
	rm -f $PIDFILE
	return "$RETVAL"
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
	#
	# If the daemon can reload its configuration without
	# restarting (for example, when it is sent a SIGHUP),
	# then implement that here.
	#
	start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
	return 0
}

case "$1" in
  start)
	[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  stop)
	[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  status)
       status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
       ;;
  #reload|force-reload)
	#
	# If do_reload() is not implemented then leave this commented out
	# and leave 'force-reload' as an alias for 'restart'.
	#
	#log_daemon_msg "Reloading $DESC" "$NAME"
	#do_reload
	#log_end_msg $?
	#;;
  restart|force-reload)
	#
	# If the "reload" option is implemented then remove the
	# 'force-reload' alias
	#
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case "$?" in
	  0|1)
		do_start
		case "$?" in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
		;;
	  *)
	  	# Failed to stop
		log_end_msg 1
		;;
	esac
	;;
  *)
	#echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
	echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
	exit 3
	;;
esac

Make the mongodb initscript executable:

sudo chmod 755 /etc/init.d/mongodb

Start mongodb. It may take a minute to start on slower machines:

sudo /etc/init.d/mongodb start

Enkive (and Postfix)

Now unpack the Enkive tarball and put it in /opt. We also create a symlink here.

sudo tar zxvf enkive-1.4.tar.gz -C /opt/
cd /opt/
sudo ln -s enkive-1.4/ enkive

Create the Enkive socket filter:

cd /opt/enkive/support/filters/postfix/enkive-socket-filter/
sudo make

This socket filter is now at:

/opt/enkive/support/filters/postfix/enkive-socket-filter/dist/enkive-socket-filter

We'll need the path to it for Postfix.

Edit the /etc/postfix/master.cf file as follows. There's already a line at the top for "smtp". You will add a second line below it for the content filter, and then 3-line filter section.

The top of the master.cf file should look something like this:

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       n       -       -       smtpd
   -o content_filter=filter:dummy
filter    unix  -       n       n       -       -      pipe
          flags=Rq user=nobody null_sender=
          argv=/opt/enkive/support/filters/postfix/enkive-socket-filter/dist/enkive-socket-filter localhost 2527 ${sender} ${recipient}
#smtp      inet  n       -       -       -       1       postscreen
#smtpd     pass  -       -       -       -       -       smtpd
#dnsblog   unix  -       -       -       -       0       dnsblog
#tlsproxy  unix  -       -       -       -       0       tlsproxy
...

Restart Postfix:

sudo /etc/init.d/postfix restart

Create an "enkive" user. The -r option will make a system user, rather than a regular user:

sudo useradd -r enkive

Allow the new "enkive" user to read and write to Enkive (don't forget the trailing slash!):

sudo chown -R enkive.enkive /opt/enkive/

Prepare the enkive initscript.

If you are using the Upstart script:

sudo cp /opt/enkive/scripts/init/ubuntu/enkive.conf /etc/init/

then edit the environment variables at the top of the script.

Otherwise, you can use the SysV initscript:

sudo cp /opt/enkive/scripts/init/debian/enkive /etc/init.d/
sudo chmod 755 /etc/init.d/enkive

At the top of the enkive initscript you need to edit several environment variables.

Now start enkive. The Upstart way:

sudo service enkive start

The SysV way:

sudo /etc/init.d/enkive start

Give it a few seconds. Then log in to a graphical desktop and use the web browser to go to http://localhost:8888/ediscovery/. Even better: from another computer on your network go to http://enkiveIPaddress:8888/ediscovery/ in a web browser.

Log in as user "enkive" with the password "enkive". There won't be any archived email yet to search. Log out.

Add email to Enkive

Sent a test email to the Enkive using the telnet client. If you've never done this before, you might want to read the following:

http://workaround.org/ispmail/lenny/test-mail-through-telnet
http://www.expta.com/2010/03/how-to-use-telnet-to-send-smtp-email-to.html (Windows-centric, but useful)

Follow the session below. You can only escape from the telnet session with CTRL-] (not CTRL-C!).

The actual lines you type after telnetting to port 25 are:

ehlo localhost
mail from:<bill@example.com>
rcpt to:<ted@example.org>
data
From: Bill <bill@example.com>
To: Ted <ted@example.org>
Subject: my first archived email

Archive me, please!

.

Note that you finally "send" the email by typing a single period by itself on a line, then hit enter. After which, postfix will queue the message.

user@localhost:~$ telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 localhost ESMTP Postfix (Ubuntu)
ehlo localhost
250-localhost
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from:<bill@example.com>
250 2.1.0 Ok
rcpt to:<ted@example.org>
250 2.1.5 Ok
data
354 End data with <CR><LF>.<CR><LF>
From: Bill <bill@example.com>
To: Ted <ted@example.org>
Subject: my first archived email

Archive me, please!

.

250 2.0.0 Ok: queued as D002D28D02
^]    
telnet> quit

Now go back to http://localhost:8888/discovery and log in. You will be able to search for the email.

Final clean up and reboot

The Upstart scripts provided in the Enkive tarball are configured so that Enkive starts when it detects that MongoDB is running.

If you ues the old style, these update-rc.d commands make sure that MongoDB starts first, and that Enkive starts well after it. See the note below re: problems with Enkive

sudo update-rc.d mongodb start 60 2 3 4 5 . stop 40 0 1 6 .
sudo update-rc.d enkive start 90 2 3 4 5 . stop 10 0 1 6 .

Reboot the machine for good measure.

Note: MongoDB should start before Enkive. If you're having issues with Enkive not starting on boot, it could be that MongoDB has not finished starting when Enkive starts. If that's the case, you need to modify the sequence numbers in the mongodb and enkive update-rc.d commands. The current settings used here should work, however.

You're mostly done

At this point, if you want to start archiving email you will have configure your mail server (and there may be more than one in your organization) to send a duplicate of each email to this Enkive server.

Postfix (including Zimbra), sendmail and exim have different ways of doing this; however, this is not covered here yet.

HOWTO install Enkive 1.4 on Ubuntu 12.04 LTS

From Enkive Wiki

Contents

Prerequisites

Preliminary steps

Postfix

Development environment

Java

Jsvc

Swig

Indri

MongoDB

Enkive (and Postfix)

Add email to Enkive

Final clean up and reboot

You're mostly done

Views

Personal tools

Navigation

Search

Toolbox