HOWTO install Enkive 1.4 on Ubuntu 12.04 LTS
From Enkive Wiki
This HOWTO documents step-by-step how to install and configure Enkive 1.4 on Ubuntu 12.04 LTS (Precise Pangolin).
Before continuing, you should read the official, and generic, installation instructions for Enkive 1.4 at http://wiki.enkive.org/index.php/1.4_Installation_Instructions.
Assumptions: This will be a standalone server that only runs the Enkive email archiver. Any email sent to this machine's postfix (port 25) gets archived. In a production environment you will have to configure your existing mail server(s) to send a copy of each email that comes in to and goes out of your organization to this Enkive server. Configuring your existing mail server(s) to send duplicate emails to Enkive is not covered in this HOWTO.
Contents |
Prerequisites
The files used to install Enkive 1.4. These were the latest available versions at the time of this writing (Feb. 2014):
- enkive-1.4.tar.gz
- jdk-7u51-linux-x64.tar.gz
- mongodb-linux-x86_64-2.4.9.tgz
- indri-5.6.tar.gz
- commons-daemon-1.0.15-native-src.tar.gz (do not use an earlier version than 1.0.12 with Enkive 1.4)
The download links are available at http://wiki.enkive.org/index.php/1.4_Installation_Instructions.
Linux operating system:
- Ubuntu 12.04 LTS Server default install.
Make sure you have at least 10GB (and preferably a lot more) of free space on your machine. MongoDB won't start properly if there's not enough free space.
Preliminary steps
Postfix
Ubuntu 12.04 does not have any mail transfer agent installed. So, install the Postfix MTA. Enkive only works with postfix at this time.
sudo apt-get install postfix
After you type this, you will get a pop-up asking what for the kind of mail configuration. Select "Internet Site". Next it will ask for the "system mail name", which you can leave as is -- whatever you already selected as your server's hostname.
Postfix will now be running, and listening on your network interface (not just localhost).
You will also need to modify the "mynetworks" line in /etc/postfix/main.cf. By default, the Ubuntu postfix will only accept mail from itself, not even the local network. So find this line:
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
and add your local subnet. For example, if your local network is in 192.168.1.0/24, change the line to read:
mynetworks = 127.0.0.0/8 192.168.1.0/24 [::ffff:127.0.0.0]/104 [::1]/128
The above assumes that the production mail server that will be sending copies of email to Enkive is on the same local network. It may be on a different subnet; for example, your mail server might have an IP address of 172.16.1.5 and the Enkive server an IP address of 10.1.1.20. Then the "mynetworks" line in main.cf might look like:
mynetworks = 127.0.0.0/8 172.16.1.0/16 10.1.1.0/24 [::ffff:127.0.0.0]/104 [::1]/128
which would accept mail from anyone on the 172.16.1.0 and 10.1.1.0 subnets.
When Enkive is finally up and running you can change the "mynetworks" line to restrict mail, allowing it from just a certain IP, for example.
After you have saved /etc/postfix/main.cf, reload postfix:
sudo /etc/init.d/postfix reload
Development environment
Some of the Enkive-related packages need to be compiled, so get a development environment installed. At minimum, the packages you'll need are:
- gcc
- g++
- make
- libcap-dev
sudo apt-get install gcc g++ make libcap-dev
Alternatively, installing the "build-essential" package will also get you most packages needed for compiling:
sudo apt-get install build-essential libcap-dev
Java
Unpack the tarball in /opt:
sudo tar zxvf jdk-7u51-linux-x64.tar.gz -C /opt/
Make a symbolic link:
cd /opt/ sudo ln -s jdk1.7.0_51/ java
So, per the official Enkive instructions, $JAVA_HOME is: /opt/jdk1.7.0_51 or /opt/java. We'll use /opt/java as the $JAVA_HOME from now on.
You may want to put /opt/java/bin in your path. There are various ways to do this. You can modify your .bashrc file, or even the /etc/environment file (which applies system-wide). Remember to log out and log back in for the changes to take effect.
Jsvc
You must use Jsvc version 1.0.12 (or later) with Enkive 1.4. (Version 1.0.15 is recommended.) Prior versions of Jsvc (1.0.11 and earlier) are not compatible with Enkive 1.4. Do not use the Jsvc from the Ubuntu 12.04 repository.
You can download previous versions of Jsvc at http://archive.apache.org/dist/commons/daemon/source/.
For Jsvc, we will compile with the java option and then copy the resulting binary to another directory so that it's in our path. Note that $JAVA_HOME is /opt/java:
tar zxvf commons-daemon-1.0.15-native-src.tar.gz cd commons-daemon-1.0.15-native-src/ cd unix/ ./configure --with-java=/opt/java make sudo cp jsvc /usr/local/bin/
Now "jsvc" is in your path. This is also where the enkive initscript expects to find the jsvc binary.
Swig
For swig:
sudo apt-get install swig zlib1g zlib1g-dev
Indri
For indri, we're going to install it in /opt/indri, as the official documentation suggests.
tar zxvf indri-5.6.tar.gz cd indri-5.6/ ./configure --prefix=/opt/indri-5.6 --enable-java --with-javahome=/opt/java make sudo make install
Create a symbolic link for /opt/indri:
cd /opt/ ln -s indri-5.6/ indri
MongoDB
There are several different ways to install MongoDB.
The easiest may be to use the custom MongoDB repository that the MongoDB maintainers offer. See the following link for how to install and configure:
http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/
(The MongoDB that is part of the Ubuntu 12.04 repository is slightly older, version 2.0.4. This will work, but we recommend using a 2.4.x MongoDB.)
If you just use the tarball provided on the downloads page -- mongodb-linux-x86_64-2.4.9.tgz -- keep in mind that it does not come with an initscript or upstart script.
An upstart script can be found in the Enkive tarball in the scripts/init/ubuntu/ subdirectory. Or you can use the SysV initscript below.
If you're using the mongodb-linux-x86_64-2.4.9.tgz tarball, unpack it in /opt. Create a symbolic link for it. Create a directory within it (ie, data/db) for the database files.
sudo tar zxvf mongodb-linux-x86_64-2.4.9.tgz -C /opt/ cd /opt/ sudo ln -s mongodb-linux-x86_64-2.4.9/ mongodb cd mongodb/ sudo mkdir -p data/db
Create the initscript for mongodb, and put the following in /etc/init.d/mongodb:
#! /bin/sh ### BEGIN INIT INFO # Provides: mongodb # Required-Start: $remote_fs $syslog $network # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: MongoDB initscript # Description: This file should be used to construct scripts to be # placed in /etc/init.d. ### END INIT INFO # Author: Foo Bar <foobar@baz.org> # # Please remove the "Author" lines above and replace them # with your own name if you copy and modify this script. # Do NOT "set -e" # PATH should only include /usr/* if it runs after the mountnfs.sh script #PATH=/sbin:/usr/sbin:/bin:/usr/bin PATH=/opt/mongodb/bin:/sbin:/usr/sbin:/bin:/usr/bin DESC="MongoDB" NAME=mongod DAEMON=/opt/mongodb/bin/$NAME #DAEMON_ARGS="--options args" DAEMON_ARGS="--fork --dbpath /opt/mongodb/data/db --logpath /var/log/mongodb.log --logappend" PIDFILE=/var/run/$NAME.pid SCRIPTNAME=/etc/init.d/mongodb # Exit if the package is not installed [ -x "$DAEMON" ] || exit 0 # Read configuration variable file if it is present [ -r /etc/default/$NAME ] && . /etc/default/$NAME # Load the VERBOSE setting and other rcS variables . /lib/init/vars.sh # Define LSB log_* functions. # Depend on lsb-base (>= 3.0-6) to ensure that this file is present. . /lib/lsb/init-functions # # Function that starts the daemon/service # do_start() { # Return # 0 if daemon has been started # 1 if daemon was already running # 2 if daemon could not be started start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \ || return 1 start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \ $DAEMON_ARGS 2>&1 >>/var/log/mongodb.log \ || return 2 # Add code here, if necessary, that waits for the process to be ready # to handle requests from services started subsequently which depend # on this one. As a last resort, sleep for some time. } # # Function that stops the daemon/service # do_stop() { # Return # 0 if daemon has been stopped # 1 if daemon was already stopped # 2 if daemon could not be stopped # other if a failure occurred start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME RETVAL="$?" [ "$RETVAL" = 2 ] && return 2 # Wait for children to finish too if this is a daemon that forks # and if the daemon is only ever run from this initscript. # If the above conditions are not satisfied then add some other code # that waits for the process to drop all resources that could be # needed by services started subsequently. A last resort is to # sleep for some time. start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON [ "$?" = 2 ] && return 2 # Many daemons don't delete their pidfiles when they exit. rm -f $PIDFILE return "$RETVAL" } # # Function that sends a SIGHUP to the daemon/service # do_reload() { # # If the daemon can reload its configuration without # restarting (for example, when it is sent a SIGHUP), # then implement that here. # start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME return 0 } case "$1" in start) [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME" do_start case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac ;; stop) [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME" do_stop case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac ;; status) status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $? ;; #reload|force-reload) # # If do_reload() is not implemented then leave this commented out # and leave 'force-reload' as an alias for 'restart'. # #log_daemon_msg "Reloading $DESC" "$NAME" #do_reload #log_end_msg $? #;; restart|force-reload) # # If the "reload" option is implemented then remove the # 'force-reload' alias # log_daemon_msg "Restarting $DESC" "$NAME" do_stop case "$?" in 0|1) do_start case "$?" in 0) log_end_msg 0 ;; 1) log_end_msg 1 ;; # Old process is still running *) log_end_msg 1 ;; # Failed to start esac ;; *) # Failed to stop log_end_msg 1 ;; esac ;; *) #echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2 echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2 exit 3 ;; esac
Make the mongodb initscript executable:
sudo chmod 755 /etc/init.d/mongodb
Start mongodb. It may take a minute to start on slower machines:
sudo /etc/init.d/mongodb start
Enkive (and Postfix)
Now unpack the Enkive tarball and put it in /opt. We also create a symlink here.
sudo tar zxvf enkive-1.4.tar.gz -C /opt/ cd /opt/ sudo ln -s enkive-1.4/ enkive
Create the Enkive socket filter:
cd /opt/enkive/support/filters/postfix/enkive-socket-filter/ sudo make
This socket filter is now at:
/opt/enkive/support/filters/postfix/enkive-socket-filter/dist/enkive-socket-filter
We'll need the path to it for Postfix.
Edit the /etc/postfix/master.cf file as follows. There's already a line at the top for "smtp". You will add a second line below it for the content filter, and then 3-line filter section.
The top of the master.cf file should look something like this:
# # Postfix master process configuration file. For details on the format # of the file, see the master(5) manual page (command: "man 5 master"). # # Do not forget to execute "postfix reload" after editing this file. # # ========================================================================== # service type private unpriv chroot wakeup maxproc command + args # (yes) (yes) (yes) (never) (100) # ========================================================================== smtp inet n - n - - smtpd -o content_filter=filter:dummy filter unix - n n - - pipe flags=Rq user=nobody null_sender= argv=/opt/enkive/support/filters/postfix/enkive-socket-filter/dist/enkive-socket-filter localhost 2527 ${sender} ${recipient} #smtp inet n - - - 1 postscreen #smtpd pass - - - - - smtpd #dnsblog unix - - - - 0 dnsblog #tlsproxy unix - - - - 0 tlsproxy ...
Restart Postfix:
sudo /etc/init.d/postfix restart
Create an "enkive" user. The -r option will make a system user, rather than a regular user:
sudo useradd -r enkive
Allow the new "enkive" user to read and write to Enkive (don't forget the trailing slash!):
sudo chown -R enkive.enkive /opt/enkive/
Prepare the enkive initscript.
If you are using the Upstart script:
sudo cp /opt/enkive/scripts/init/ubuntu/enkive.conf /etc/init/
then edit the environment variables at the top of the script.
Otherwise, you can use the SysV initscript:
sudo cp /opt/enkive/scripts/init/debian/enkive /etc/init.d/ sudo chmod 755 /etc/init.d/enkive
At the top of the enkive initscript you need to edit several environment variables.
Now start enkive. The Upstart way:
sudo service enkive start
The SysV way:
sudo /etc/init.d/enkive start
Give it a few seconds. Then log in to a graphical desktop and use the web browser to go to http://localhost:8888/ediscovery/. Even better: from another computer on your network go to http://enkiveIPaddress:8888/ediscovery/ in a web browser.
Log in as user "enkive" with the password "enkive". There won't be any archived email yet to search. Log out.
Add email to Enkive
Sent a test email to the Enkive using the telnet client. If you've never done this before, you might want to read the following:
- http://workaround.org/ispmail/lenny/test-mail-through-telnet
- http://www.expta.com/2010/03/how-to-use-telnet-to-send-smtp-email-to.html (Windows-centric, but useful)
Follow the session below. You can only escape from the telnet session with CTRL-] (not CTRL-C!).
The actual lines you type after telnetting to port 25 are:
ehlo localhost mail from:<bill@example.com> rcpt to:<ted@example.org> data From: Bill <bill@example.com> To: Ted <ted@example.org> Subject: my first archived email Archive me, please! .
Note that you finally "send" the email by typing a single period by itself on a line, then hit enter. After which, postfix will queue the message.
user@localhost:~$ telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 localhost ESMTP Postfix (Ubuntu) ehlo localhost 250-localhost 250-PIPELINING 250-SIZE 10240000 250-VRFY 250-ETRN 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-8BITMIME 250 DSN mail from:<bill@example.com> 250 2.1.0 Ok rcpt to:<ted@example.org> 250 2.1.5 Ok data 354 End data with <CR><LF>.<CR><LF> From: Bill <bill@example.com> To: Ted <ted@example.org> Subject: my first archived email Archive me, please! . 250 2.0.0 Ok: queued as D002D28D02 ^] telnet> quit
Now go back to http://localhost:8888/discovery and log in. You will be able to search for the email.
Final clean up and reboot
The Upstart scripts provided in the Enkive tarball are configured so that Enkive starts when it detects that MongoDB is running.
If you ues the old style, these update-rc.d commands make sure that MongoDB starts first, and that Enkive starts well after it. See the note below re: problems with Enkive
sudo update-rc.d mongodb start 60 2 3 4 5 . stop 40 0 1 6 . sudo update-rc.d enkive start 90 2 3 4 5 . stop 10 0 1 6 .
Reboot the machine for good measure.
Note: MongoDB should start before Enkive. If you're having issues with Enkive not starting on boot, it could be that MongoDB has not finished starting when Enkive starts. If that's the case, you need to modify the sequence numbers in the mongodb and enkive update-rc.d commands. The current settings used here should work, however.
You're mostly done
At this point, if you want to start archiving email you will have configure your mail server (and there may be more than one in your organization) to send a duplicate of each email to this Enkive server.
Postfix (including Zimbra), sendmail and exim have different ways of doing this; however, this is not covered here yet.