httpry – is a specialized packet sniffer designed for displaying and logging HTTP traffic

httpry

Current version: 0.1.5

dumpster / jason / httpry


core program

httpry is a specialized packet sniffer designed for displaying and logging HTTP traffic. It is not intended to perform analysis itself, but to capture, parse, and log the traffic for later analysis. It can be run in real-time displaying the traffic as it is parsed, or as a daemon process that logs to an output file. It is written to be as lightweight and flexible as possible, so that it can be easily adaptable to different applications.

What can you do with it? Here’s a few ideas:

  • See what users on your network are requesting online
  • Check for proper server configuration (or improper, as the case may be)
  • Research patterns in HTTP usage
  • Watch for dangerous downloaded files
  • Verify the enforcement of HTTP policy on your network
  • Extract HTTP statistics out of saved capture files
  • It’s just plain fun to watch in realtime

Here’s an example of the log file output using the default output format string:

# httpry version 0.1.5
# Fields: timestamp,source-ip,dest-ip,direction,method,host,request-uri,http-version,status-code,reason-phrase
2009-01-12 15:02:31 192.168.0.16 209.85.171.103 > GET http://www.google.com / HTTP/1.1 – –
2009-01-12 15:02:31 192.168.0.16 209.85.171.103 > GET http://www.google.com / HTTP/1.1 – –
2009-01-12 15:02:32 192.168.0.16 209.85.171.103 > GET http://www.google.com / HTTP/1.1 – –
2009-01-12 15:02:33 192.168.0.16 209.85.171.103 > GET http://www.google.com / HTTP/1.1 – –
2009-01-12 15:02:33 209.85.171.103 192.168.0.16 < – – – HTTP/1.1 200 OK
2009-01-12 15:02:33 192.168.0.16 209.85.171.103 > GET http://www.google.com /intl/en_ALL/images/logo.gif HTTP/1.1 – –
2009-01-12 15:02:33 209.85.171.103 192.168.0.16 < – – – HTTP/1.1 200 OK
2009-01-12 15:02:33 192.168.0.16 209.85.171.103 > GET http://www.google.com /extern_js/f/CgJlbhICdXMrMAo4DSwrMA44AywrMBg4Ayw/AQ-hC7_2R8g.js HTTP/1.1 – –
2009-01-12 15:02:33 209.85.171.103 192.168.0.16 < – – – HTTP/1.1 200 OK
2009-01-12 15:02:33 192.168.0.16 209.85.173.101 > GET clients1.google.com /generate_204 HTTP/1.1 – –
2009-01-12 15:02:33 209.85.173.101 192.168.0.16 < – – – HTTP/1.1 204 No Content

parsing scripts

Of course, the fun of collecting data is finding ways to analyze it. The log files are designed to be easily parsed by command line utilities, but sometimes you need to dig a little deeper. Complementing the core httpry program is a set of parsing scripts for mining information out of generated log files. Most of these scripts are written as plugins for a core parsing script and include functionality for extracting search terms, searching for specified terms within client flows, and outputting the logs in XML among other things. It is relatively straightforward to write custom plugins for additional parsing tasks.

latest news

There are several major enhancements in the latest version of httpry. The two major added features are SIGHUP handling for gracefully reopening output files (thanks to Philipp for providing a patch with this functionality) and defaulting output files to line buffering (partly related to the same patch, but also requested by several other users). In addition, there is a new binary pcap dump file option, and both “source-port” and “dest-port” were added as available output fields.

Important Note! The command line switch for specifying a format string was changed from -s to -f to make it more mnemonic. If you have any scripts that use this switch, be sure to update them accordingly.

The Perl log parsing scripts were also updated significantly for this release. Across all plugins, the init() and end() calls are now executed with eval() to make them more resilient to errors. There is also a new list() function added to each plugin that allows it to specify required fields in the input file. If the fields aren’t present then the plugin will be disabled. The content analysis plugin was also substantially rewritten to use a sliding window to specify flows instead of time delimiting them.

Download httpry 0.1.5

If you are using FreeBSD, you can also get httpry as a FreeBSD port.


Deixe uma resposta

Preencha os seus dados abaixo ou clique em um ícone para log in:

Logotipo do WordPress.com

Você está comentando utilizando sua conta WordPress.com. Sair / Alterar )

Imagem do Twitter

Você está comentando utilizando sua conta Twitter. Sair / Alterar )

Foto do Facebook

Você está comentando utilizando sua conta Facebook. Sair / Alterar )

Foto do Google+

Você está comentando utilizando sua conta Google+. Sair / Alterar )

Conectando a %s