HOWTO install Enkive 1.2 on Ubuntu 10.04 LTS

From Enkive Wiki

Jump to: navigation, search

This HOWTO documents step-by-step how to install and configure Enkive 1.2 on Ubuntu 10.04 LTS (Lucid Lynx).

Before continuing, you should read the official, and generic, installation instructions for Enkive 1.2 at http://wiki.enkive.org/index.php/1.2_Installation_Instructions.


Assumptions: This will be a standalone server that only runs the Enkive email archiver. Any email sent to this machine's postfix (port 25) gets archived. In a production environment you will have to configure your existing mail server(s) to send a copy of each email that comes in to and goes out of your organization to this Enkive server. Configuring your existing mail server(s) to send duplicate emails to Enkive is not covered in this HOWTO.


Contents

Prerequisites

The files used to install Enkive 1.2. These were the latest available versions at the time of this writing (Nov. 2012):

  • enkive-1.2-CE.tar.gz
  • jdk-7u9-linux-x64.tar.gz
  • mongodb-linux-x86_64-2.2.1.tgz
  • indri-5.3.tar.gz
  • commons-daemon-1.0.10-native-src.tar.gz (do not use a later version of this with Enkive 1.2)

The download links are available at http://wiki.enkive.org/index.php/1.2_Installation_Instructions.

Linux operating system:

  • Ubuntu 10.04 LTS. Default graphical install.


Make sure you have at least 10GB (and preferably a lot more) of free space on your machine. Mongodb won't start properly if there's not enough free space.


Preliminary steps

Postfix

Ubuntu 12.04 does not have any mail transfer agent installed. So, install the Postfix MTA. Enkive only works with postfix at this time.

sudo apt-get install postfix

After you type this, you will get a pop-up asking what for the kind of mail configuration. Select "Internet Site". Next it will ask for the "system mail name", which you can leave as is -- whatever you already selected as your server's hostname.

Postfix will now be running, and listening on your network interface (not just localhost).

You will also need to modify the "mynetworks" line in /etc/postfix/main.cf. By default, the Ubuntu postfix will only accept mail from itself, not even the local network. So find this line:

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

and add your local subnet. For example, if your local network is in 192.168.1.0/24, change the line to read:

mynetworks = 127.0.0.0/8 192.168.1.0/24 [::ffff:127.0.0.0]/104 [::1]/128

The above assumes that the production mail server that will be sending copies of email to Enkive is on the same local network. It may be on a different subnet; for example, your mail server might have an IP address of 172.16.1.5 and the Enkive server an IP address of 10.1.1.20. Then the "mynetworks" line in main.cf might look like:

mynetworks = 127.0.0.0/8 172.16.1.0/16 10.1.1.0/24 [::ffff:127.0.0.0]/104 [::1]/128

which would accept mail from anyone on the 172.16.1.0 and 10.1.1.0 subnets.

When Enkive is finally up and running you can change the "mynetworks" line to restrict mail, allowing it from just a certain IP, for example.

After you have saved /etc/postfix/main.cf, reload postfix:

sudo /etc/init.d/postfix reload


Development environment

Some of the Enkive-related packages need to be compiled, so get a development environment installed. At minimum, the packages you'll need are:

  • gcc
  • g++
  • make
  • libcap-dev
sudo apt-get install gcc g++ make libcap-dev

Alternatively, installing the "build-essential" package will also get you most packages needed for compiling:

sudo apt-get install build-essential libcap-dev


Java

Unpack the tarball in /opt:

sudo tar zxvf jdk-7u9-linux-x64.tar.gz -C /opt/

Make a symbolic link:

cd /opt/
sudo ln -s jdk1.7.0_09/ java

So, per the official Enkive instructions, $JAVA_HOME is: /opt/jdk1.7.0_09 or /opt/java. We'll use /opt/java as the $JAVA_HOME from now on.

You may want to put /opt/java/bin in your path. There are various ways to do this. You can modify your .bashrc file, or even the /etc/environment file (which applies system-wide). Remember to log out and log back in for the changes to take effect.



Jsvc

You must use Jsvc version 1.0.10 (or earlier) with Enkive 1.2. Later versions of Jsvc (1.0.11 and later) are not compatible with Enkive 1.2.

For Jsvc, we will compile with the java option and then copy the resulting binary to another directory so that it's in our path. Note that $JAVA_HOME is /opt/java:

tar zxvf commons-daemon-1.0.10-native-src.tar.gz
cd commons-daemon-1.0.10-native-src/
cd unix/
./configure --with-java=/opt/java
make
sudo cp jsvc /usr/local/bin/

Now "jsvc" is in your path. This is also where the enkive initscript expects to find the jsvc binary.


Swig

For swig:

sudo apt-get install swig zlib1g zlib1g-dev


Indri

For indri, we're going to install it in /opt/indri, as the official documentation suggests.

tar zxvf indri-5.3.tar.gz
cd indri-5.3/
./configure --prefix=/opt/indri --enable-java --with-javahome=/opt/java
make
sudo make install


MongoDB

For mongodb, unpack it in /opt. Create a symbolic link for it. Create a directory within it (ie, data/db) for the database files.

sudo tar zxvf mongodb-linux-x86_64-2.2.1.tgz -C /opt/
cd /opt/
sudo ln -s mongodb-linux-x86_64-2.2.1/ mongodb
cd mongodb/
sudo mkdir -p data/db


Create the initscript for mongodb, and put the following in /etc/init.d/mongodb:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          mongodb
# Required-Start:    $remote_fs $syslog $network
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: MongoDB initscript
# Description:       This file should be used to construct scripts to be
#                    placed in /etc/init.d.
### END INIT INFO

# Author: Foo Bar <foobar@baz.org>
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.

# Do NOT "set -e"

# PATH should only include /usr/* if it runs after the mountnfs.sh script
#PATH=/sbin:/usr/sbin:/bin:/usr/bin
PATH=/opt/mongodb/bin:/sbin:/usr/sbin:/bin:/usr/bin
DESC="MongoDB"
NAME=mongod
DAEMON=/opt/mongodb/bin/$NAME
#DAEMON_ARGS="--options args"
DAEMON_ARGS="--fork --dbpath /opt/mongodb/data/db --logpath /var/log/mongodb.log --logappend"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/mongodb

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
	# Return
	#   0 if daemon has been started
	#   1 if daemon was already running
	#   2 if daemon could not be started
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
		|| return 1
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
		$DAEMON_ARGS 2>&1 >>/var/log/mongodb.log \
		|| return 2
	# Add code here, if necessary, that waits for the process to be ready
	# to handle requests from services started subsequently which depend
	# on this one.  As a last resort, sleep for some time.
}

#
# Function that stops the daemon/service
#
do_stop()
{
	# Return
	#   0 if daemon has been stopped
	#   1 if daemon was already stopped
	#   2 if daemon could not be stopped
	#   other if a failure occurred
	start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
	RETVAL="$?"
	[ "$RETVAL" = 2 ] && return 2
	# Wait for children to finish too if this is a daemon that forks
	# and if the daemon is only ever run from this initscript.
	# If the above conditions are not satisfied then add some other code
	# that waits for the process to drop all resources that could be
	# needed by services started subsequently.  A last resort is to
	# sleep for some time.
	start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
	[ "$?" = 2 ] && return 2
	# Many daemons don't delete their pidfiles when they exit.
	rm -f $PIDFILE
	return "$RETVAL"
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
	#
	# If the daemon can reload its configuration without
	# restarting (for example, when it is sent a SIGHUP),
	# then implement that here.
	#
	start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
	return 0
}

case "$1" in
  start)
	[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  stop)
	[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  status)
       status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
       ;;
  #reload|force-reload)
	#
	# If do_reload() is not implemented then leave this commented out
	# and leave 'force-reload' as an alias for 'restart'.
	#
	#log_daemon_msg "Reloading $DESC" "$NAME"
	#do_reload
	#log_end_msg $?
	#;;
  restart|force-reload)
	#
	# If the "reload" option is implemented then remove the
	# 'force-reload' alias
	#
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case "$?" in
	  0|1)
		do_start
		case "$?" in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
		;;
	  *)
	  	# Failed to stop
		log_end_msg 1
		;;
	esac
	;;
  *)
	#echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
	echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
	exit 3
	;;
esac

:


Make the mongodb initscript executable:

sudo chmod 755 /etc/init.d/mongodb

Start mongodb. It may take a minute to start on slower machines:

sudo /etc/init.d/mongodb start


Enkive (and Postfix)

Now unpack the Enkive tarball and put it in /opt. We also create a symlink here.

sudo tar zxvf enkive-1.2-CE.tar.gz -C /opt/
cd /opt/
sudo ln -s enkive-1.2-CE/ enkive


Create the Enkive socket filter:

cd /opt/enkive/support/filters/postfix/enkive-socket-filter/
sudo make

This socket filter is now at:

/opt/enkive/support/filters/postfix/enkive-socket-filter/dist/enkive-socket-filter 

We'll need the path to it for Postfix.

Edit the /etc/postfix/master.cf file as follows. There's already a line at the top for "smtp". You will add a second line below it for the content filter, and then 3-line filter section.

The top of the master.cf file should look something like this:

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
smtp      inet  n       -       n       -       -       smtpd
   -o content_filter=filter:dummy
filter    unix  -       n       n       -       -      pipe
          flags=Rq user=nobody null_sender=
          argv=/opt/enkive/support/filters/postfix/enkive-socket-filter/dist/enkive-socket-filter localhost 2527 ${sender} ${recipient}
#submission inet n       -       -       -       -       smtpd
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
...

Restart Postfix:

sudo /etc/init.d/postfix restart


Copy the indri.jar file from /opt/indri/share/indri/ to /opt/enkive/lib/:

sudo cp -p /opt/enkive/lib/indri.jar /opt/enkive/lib/indri.jar.orig
sudo cp /opt/indri/share/indri/indri.jar /opt/enkive/lib/


Create an "enkive" user. The -r option will make a system user, rather than a regular user:

sudo useradd -r enkive


Allow the new "enkive" user to read and write to Enkive (don't forget the trailing slash!):

sudo chown -R enkive.enkive /opt/enkive/

Prepare the enkive initscript.

sudo cp /opt/enkive/scripts/enkive /etc/init.d/enkive
sudo chmod 755 /etc/init.d/enkive


At the top of the enkive initscript you need to edit several variables.

The relevant portion of the initscript:

...
# SYSTEM ADMINISTRATOR LIKELY WANTS TO MODIFY THE FOLLOWING VARIABLES

# user id under which Enkive is run; a good practice is to create a user specifically to run Enkive (e.g., "enkive")
ENKIVE_USER=enkive

# path to top-level Enkive directory; this directory would contain "lib" and "config"
#ENKIVE_HOME=$(pwd)
ENKIVE_HOME=/opt/enkive

# path to the jsvc executable file
JSVC=/usr/local/bin/jsvc

# path to top level of the Java JDK (must be a JDK and not a JRE); should contain "bin", "lib", and "lib/tools.jar"
#JAVA_HOME=/usr/lib/jvm/java-openjdk
JAVA_HOME=/opt/java

# path to directory that contains file "libindri_jni.so"
#INDRI_SO_PATH=/usr/local/lib
INDRI_SO_PATH=/opt/indri/lib
...


Now start enkive:

sudo /etc/init.d/enkive start

Give it a few seconds. Then log in to a graphical desktop and use the web browser to go to http://localhost:8888/ediscovery/. Even better: from another computer on your network go to http://enkiveIPaddress:8888/ediscovery/ in a web browser.

Log in as user "enkive" with the password "enkive". There won't be any archived email yet to search. Log out.


Add email to Enkive

Sent a test email to the Enkive using the telnet client. If you've never done this before, you might want to read the following:

Follow the session below. You can only escape from the telnet session with CTRL-] (not CTRL-C!).

The actual lines you type after telnetting to port 25 are:

ehlo localhost
mail from:<bill@example.com>
rcpt to:<ted@example.org>
data
From: Bill <bill@example.com>
To: Ted <ted@example.org>
Subject: my first archived email

Archive me, please!

.


Note that you finally "send" the email by typing a single period by itself on a line, then hit enter. After which, postfix will queue the message.

user@localhost:~$ telnet localhost 25
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 localhost ESMTP Postfix (Ubuntu)
ehlo localhost
250-localhost
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from:<bill@example.com>
250 2.1.0 Ok
rcpt to:<ted@example.org>
250 2.1.5 Ok
data
354 End data with <CR><LF>.<CR><LF>
From: Bill <bill@example.com>
To: Ted <ted@example.org>
Subject: my first archived email

Archive me, please!

.
250 2.0.0 Ok: queued as C9FFDA3A55
^]

telnet> 

Now go back to http://localhost:8888/discovery and log in. You will be able to search for the email.


Final clean up and reboot

Make sure everything starts on boot. These update-rc.d commands make sure that MongoDB starts first, and that Enkive starts well after it. See the note below re: problems with Enkive

sudo update-rc.d mongodb start 60 2 3 4 5 . stop 40 0 1 6 .
sudo update-rc.d enkive start 90 2 3 4 5 . stop 10 0 1 6 .


Reboot the machine for good measure.

Note: MongoDB should start before Enkive. If you're having issues with Enkive not starting on boot, it could be that MongoDB has not finished starting when Enkive starts. If that's the case, you need to modify the sequence numbers in the mongodb and enkive update-rc.d commands. The current settings used here should work, however.



You're mostly done

At this point, if you want to start archiving email you will have configure your mail server (and there may be more than one in your organization) to send a duplicate of each email to this Enkive server.

Postfix (including Zimbra), sendmail and exim have different ways of doing this; however, this is not covered here yet.

Personal tools