===== IPPROT =====

A small proxy that records tcp network connections and/or any network traffic.

The ipprotd daemon listens on a port for all configured ip addresses 
or only for one specific ip address 
and connects to a destination port and ip address. All traffic is 
transparently forwarded to the destination and returned back to the source.

The ipprotd is part of the [[uls:agents:linux]].


<!-- http://graphviz-dev.appspot.com/  -->
<!-- http://sandbox.kidstrythisathome.com/erdos/  -->

<graphviz dot left width=700>

  digraph ipprot {
    node [shape=box, color=grey, fillcolor="#F6F6F6", fontcolor=black, fontsize=12];
    edge [color="#F6EA48", fontcolor="#A6A6A6", fontsize=8];

    rankdir=LR;

    connector   [label="connection\nestablisher"];

    subgraph cluster_1 {
      label="server";
      labeljust="l";
      fontcolor=azure4;
      color=grey;

      listen      [label="<listen_ip>:<port>", fontsize=8, fontcolor=darkgreen];
      ipprotd     [shape=ellipse, label="ipprotd"];
      destination [label = "<destination>:<port>", fontsize=8, fontcolor=darkgreen];
      recording   [shape=note, label="network traffic\nrecording"];
      connectinfo [shape=note, label="connection\ninformation\nlog file"];
      application [label="application"];
      process     [shape=ellipse, label="post\nprocessing\ncommand"];

      {rank=same; ipprotd; recording;}
    }
    connector -> listen -> ipprotd -> destination -> application [dir=both];
    ipprotd -> recording -> process;
    ipprotd -> connectinfo;

  }

</graphviz>

------


The ipprotd may collect all **connection information** 
like ip addresses or timestamp of begin and end of the connection 
is written to **one** log file (option [[ipprot#-c <connection_log_file>]]).

**Network traffic** (option [[ipprot#-f <data_log_file>]]) can be recorded in connection specific log files 
for incoming, for outgoing or for data in both directions. 
The traffic for each connection is written to its **own** log file.
The log file name will contain date and time, ip addresses and ports to make
it unique. After the connection is closed, a post-processing command 
may be executed (option [[ipprot#-u <cmd>]]). That receives among other things the 
log file name as a parameter.

Each connection has its own ipprotd process which terminates when the connection ends.

Use the correct operating system user to start the ipprot.

<code bash>

ipprotd [-p inport[@listen_host]] [-h out_host] [-P out_port[@out_host]] [-b buffsize]
        [-f data_log_file] [-l|-L] [-s] [-z|-j] [-t secs] [-M] [-u[T] cmd]
        [-c connection_log_file] [-C connection_log_format] 
        [-r client_ip1 [-r client_ip2 ...]]
        [-D|-Dp pid_file]
        [-?|-v]
</code>

-----

==== Options ====

Have also a look at the [[ipprot#Examples]].


=== -b <buffsize> ===

The default buffer size is 300000 Bytes, that may be to large for some szenarios, 
e.g. a client may expect a specific or maximum amount of bytes.
Then specify a smaller <buffsize> in Bytes (the buffer is always placed in memory).


=== -c <connection_log_file> ===

Write connection infos to the <connection_log_file>. 
All entries will be appended to the <connection_log_file>
when the connection is closed.
Use option [[ipprot#-C <connection_log_format>]] to specify what is written to the <connection_log_file>.
The network traffic is **not** recorded.


=== -C <connection_log_format> ===

Specify the format of a text expression that is appended 
to the <connection_log_file>. Any characters are copied unchanged into the output string, 
except the following special expressions, which are replaced by gathered connection metrics.
Only in combination with option [[ipprot#-c <connection_log_file>]].

{|
! expression 
! is replaced by
|-
| %%
| %
|-
| %e
| date and time of connection end in RFC 3339 format ("yyyy-mm-dd HH:MM:SS+hh:mm")
|-
| %i
| ip address of connection establisher
|-
| %n
| inserts a newline in the <connection_log_file> 
|-
| %P
| complete path to the <connection_log_file> 
|-
| %r
| bytes received from connection establisher
|-
| %s
| date and time of connection start in RFC 3339 format ("yyyy-mm-dd HH:MM:SS+hh:mm")
|-
| %S
| duration of connection in seconds
|-
| %t
| bytes sent to connection establisher
|-
| %u
| start timestamp of the connection in seconds since Unix epoch
|-
| %U
| end timestamp of the connection in seconds since Unix epoch
|}

Example: 
<code bash>
-C "Connected: %s, disconnected after %S secs at: %e"
</code>


=== -D ===

Start ipprotd as daemon.


=== -Dp <pid_file> ===

Start as daemon and write its pid to the specified full path of <pid_file>.


=== -f <data_log_file> ===

Specify the path and file prefix as the <data_log_file> which will contain the logged data. 
The timestamp of the start of the connection in the format yyyy-mm-dd_HHMMSS, 
a consecutive number <n> and 
the client ip <client_ip> 
is appended to the final log file name to make it unique.

Format: <data_log_file>_<yyyy-mm-dd_HHMMSS>_<n>_<client_ip>


=== -h <out_host> ===

Forward all incoming data to this <out_host>. Default is localhost.


=== -j ===

Compress the <data_log_file> by using bzip2 while it is written. 
Only in combination with option [[ipprot#-f <data_log_file>]].


=== -l ===

Log all network traffic in both directions from connection establisher, 
if neither option [[ipprot#-l]], nor option [[ipprot#-L]] is given, 
only network traffic **to** the connection establisher is recorded.
Only in combination with option [[ipprot#-f <data_log_file>]].


=== -L ===

Log all incoming network traffic from connection establisher, 
if neither option [[ipprot#-l]], nor option [[ipprot#-L]] is given, 
only network traffic **to** the connection establisher is recorded.
Only in combination with option [[ipprot#-f <data_log_file>]].

=== -M ===

(New in version 2.5) 

Each logged block is preceded by a timestamp. The timestamp 
is written as microseconds ($\mu sec$) since Unix epoch preceded by a magic number enclosed in '#'.
Only in combination with option [[ipprot#-f <data_log_file>]].

The timestamp is preceded by magic number 
  * #848367# for incoming data blocks and 
  * #948366# for outgoing data blocks. 

Examples:
  #848367#1386169176610141
  #948366#1386169176610343

You may convert that to a human readable form on most linuxes (remember that the timestamp is in microseconds):

  $ date -d@1386169176 '+%F %T'
  2013-12-04 15:59:36

Remember that magic number #748369# indicates an iso timestamp which is inserted at regular intervals, see [[ipprot#-t <secs>]].

<file - example.log>
#848367#1386170322361458

#748369#2013-12-04 16:18:42-01:00

#948366#1386170322404186
HTTP/1.1 200 OK
Date: Wed, 04 Dec 2013 15:18:42 GMT
Server: unix2web/9.1.1
Expires: Wed, 04 Dec 2013 15:18:42 GMT
Last-Modified: Wed, 04 Dec 2013 15:18:42 GMT
Content-Type: text/html; charset=UTF-8

#948366#1386170322406047
Connection: Keep-Alive
Content-Length: 535

<!DOCTYPE html>
<HTML><HEAD><meta http-equiv="Pragma" content="no-cache"><meta http-equiv="Expires" content="now"><TITLE>ULS - lulsed001</TITLE>
<link rel="stylesheet" type="text/css" href="stylesheet_uls.css">
<frameset rows="90,*"><frame name="mtop" src="kopf.u2w"><frameset cols="235, *"><frame src="index.u2w?u645342me=on847629" name="mleft"><frame src="index.u2w?u645342me=mr847629" name="mright"></frameset><noframes><body>Diese Seite verwendet Frames, der Browser unterstützt keine Frames!</body></noframes></frameset>
</HTML>

#848367#1386170322763181

#948366#1386170322767464
HTTP/1.1 200 OK
Date: Wed, 04 Dec 2013 15:18:42 GMT
Server: unix2web/9.1.1
Expires: Wed, 04 Dec 2013 15:18:42 GMT
Last-Modified: Wed, 04 Dec 2013 15:18:42 GMT
Content-Type: text/html; charset=UTF-8

#948366#1386170322790097
Connection: Keep-Alive
Content-Length: 1061

<!DOCTYPE html>
<HTML><HEAD><meta http-equiv="Pragma" content="no-cache"><meta http-equiv="Expires" content="now"><TITLE></TITLE>
<style type="text/css">
a img
{ border: 0;
}
</style>
<link rel="stylesheet" type="text/css" href="stylesheet_uls.css">
</HEAD>
<BODY>
<div><div style="float:left; text-align:left;">
<div id="mainmenue">
<A href="index.u2w?u645342me=on847629" target="mleft">Hauptmenü</A>
<A href="list_favoriten.u2w" target="mleft">Favoriten</A>
<A href="list_reports.u2w" target="mleft">Berichte</A>
<A href="index_admin.u2w" target="mleft">Verwalten</A>
<A href="operator/index.u2w" target="_top">Operator Menü</A>
<A href="admin/index.u2w" target="_top">Admin-Menü</A>
<A href="search_server.u2w?sdat=sinceyesterday" target="mleft">Server suchen</A>
</div>
<BR>
<BR>
<div id="breadcrumbs">

</div>
</div><div class="logo" style="float:right">
<A href="http://www.universal-logging-system.org/" target="_blank"><img src="ULS-0003DE-02W_ULS_Logo.png"></A><font size="4"><sup>@</sup></font><img src="Head_Logo.gif">
</div></div>
</BODY></HTML>
/dev/shm>

</file>
=== -p <inport>[@<listen_host>] ===

ipprot listens on this port on all ip addresses or only for 
<listen_host> for incoming connections and data. 
Default port is 8080 on all currently defined ip addresses
and on all additionally defined ip addresses during the 
runtime of ipprot.


=== -P <out_port>[@out_host] ===
  
Forward all incoming data to this <out_port> and <out_host>, 
which must be given as direct parameter or must be set as option [[ipprot#-h <out_host>]]. 
Default is port 80 on localhost.


=== -r <client_ip> ===

Restrict a connection for this allowed ip address. You may repeat this option for up to 30 times.


=== -s ===

Log only printable characters. CR, LF, members of the 
character classes [:blank:] and [:graph:] are logged, 
one blank is logged for any number of successive non-printable characters.
Only in combination with option [[ipprot#-f <data_log_file>]].


=== -t <secs> ===

Specify the number of seconds after which a timestamp is inserted into 
the <data_log_file> when new data arrives from the connection establisher.
Only in combination with option [[ipprot#-f <data_log_file>]].


=== -u <cmd> ===

Call <cmd> for post-processing as soon as the connection is closed.
The <cmd> will receive as parameters:

  - the full path and file name of the <data_log_file>
  - start timestamp of the connection in seconds since Unix epoch
  - end timestamp of the connection in seconds since Unix epoch
  - the number of bytes received from the connection establisher
  - the number of bytes sent to the connection establisher
  - the ip address of the connection establisher

Only in combination with option [[ipprot#-f <data_log_file>]].


=== -uT <cmd> ===

(New in version 2.4) 

Call <cmd> for post-processing as soon as the connection is closed.
The <cmd> will receive as parameters:

  - the full path and file name of the <data_log_file>
  - start timestamp of the connection in the RFC 3339 format yyyy-mm-dd HH:MM:SS+hh:mm
  - end timestamp of the connection in the RFC 3339 format yyyy-mm-dd HH:MM:SS+hh:mm
  - the number of bytes received from the connection establisher
  - the number of bytes sent to the connection establisher
  - the ip address of the connection establisher

Only in combination with option [[ipprot#-f <data_log_file>]].


=== -v ===

Show version information.


=== -z ===

Compress the <data_log_file> by using gzip while it is written.
Only in combination with option [[ipprot#-f <data_log_file>]].


=== -? ===

Show this help.



==== Examples ====

Here are two examples of ipprot's usage and their explanations.


=== Example 1 ===

Start ipprotd as daemon and 

  * listen on port 5432 for all locally configured ip addresses and all possibly later dynamically configured ip addresses, 
  * forward the connection to ip 127.0.0.1, port 3333, 
  * record network traffic in both directions, 
  * write recorded network traffic to file /var/log/conn_rec/prot6543_<dest_ip>_<yyyy-mm-dd_HHMMSS>_<listen_ip>, the <values> will be replaced by the actual values of the connection or timestamp,
  * gzip the file during writing,
  * write the pid of the ipprotd master process to file /var/log/conn_rec/ipprot_pid_5432

<code bash>
$ ipprotd -p 5432 -h 127.0.0.1 -P 3333 
          -l -f /var/log/conn_rec/prot -z 
          -Dp /var/log/conn_rec/ipprot_pid_5432
</code>

Stop thru kill:

<code bash>
$ kill `cat /var/log/conn_rec/ipprot_pid_5432`
</code>

Currently active ipprotd processes will continue to execute 
until the connection of each respective process is closed normally.


=== Example 2 ===

A typical security requirement is the recording of all sql commands
that are executed by a database administrator 
usind e.g. the SQL Developer from his work station. (Opposed to 
a local connection to the database as sysdba, which does not use 
the listener and therefor cannot be captured by ipprot. See 
"Logging a Terminal Session" in the "ULS-client for Linux" 
documentation to find out how to do that.

Start ipprotd as daemon and 

  * listen on ip 10.1.2.33, port 7777
  * forward the connection to ip 10.1.2.34, port 5678, 
  * record only printable incoming network traffic, 
  * write recorded network traffic to file /u01/oradata/orcl/connection_protocol/prot_7777_10.1.2.34_<yyyy-mm-dd_HHMMSS>_10.1.2.33,
  * bzip2 that file during it is written,
  * call post-processing script /u01/oradata/orcl/oracle_tools/send_ipprot after the connection has been closed and bzip2 has terminated,
  * write the pid of the ipprotd master process to file /u01/oradata/orcl/connection_protocol/pid_7777

<code bash>
$ ipprotd -p 7777@10.1.2.33 -P 5678@10.1.2.34 
          -L -s -f /u01/oradata/orcl/connection_protocol/prot -j  
          -u /u01/oradata/orcl/oracle_tools/send_ipprot 
          -Dp /u01/oradata/orcl/connection_protocol/pid_7777
</code>

Stop thru kill:

<code bash>
$ kill `cat /u01/oradata/orcl/connection_protocol/pid_7777`
</code>

Currently active ipprotd processes will continue to execute 
until the connection of each respective process is closed normally.


==== Installation ====

The ipprotd is part of the [[uls:agents:linux]].