Libp2p

From Phoenix Labs Wiki

libp2p is the cross-platform C++ library that PeerGuardian 2 uses to load lists. libp2p is capable of loading lists of the P2P, P2B, and eMule .dat formats, and provides highly optimized algorithms for merging, subtracting, and optimizing lists.

1 Reference
2 Example
3 Open Source
4 See also
5 External links

Reference

Unless explicitly shown, everything is within the p2p namespace.

Ranges

struct range provides basic operations for single ranges.

#include <string>

struct range {
   std::wstring name;
   ip start, end;

   range();
   range(const std::wstring &name);
   range(const std::wstring &name, unsigned int start, unsigned int end);

   bool operator<(const range &range) const;
   bool operator>(const range &range) const;
   bool operator==(const range &range) const;
}

Lists

class list is the heart of the library, providing algorithms for inserting, removing, merging, and optimizing of lists. list is optimized for manipulating lists - once you are done building a list, you should construct a static_list or compact_list to test IPs against.

#include <list>
#include "p2p/compact_list.hpp"

class list {
public:
   typedef std::list<range> list_type;
   typedef list_type::size_type size_type;
   typedef list_type::iterator iterator;
   typedef list_type::const_iterator const_iterator;

   void insert(const range &r);
   void insert(const list &l);
   void erase(const range &r);
   void erase(const compact_list &l);

   void splice(list &l);

   void optimize(bool aggressive = false);

   iterator begin();
   iterator end();

   const_iterator begin() const;
   const_iterator end() const;

   size_type size() const;
   void clear();
};

void insert(const range &r): Inserts a range at the end of the list.

void insert(const list &l): Concatenates a list to the end of the list.

void erase(const range &r): Erases all IPs which are within a range.

void erase(const compact_list &l): Erases all IPs which are in a compact_list. class list is implicitly convertable to class compact_list.

void splice(list &l): Splices a list to the end of the list. No copying occurs: ranges from the passed list are moved directly into the list.; If you want to concatentate a list and have no use for the passed list after it is merged, splice is much faster than insert.

void optimize(bool aggressive = false): Optimizes the list by removing overlapping ranges and merging adjacent ones.; If aggressive is false, adjacent ranges will not be merged if their names differ.; If aggressive is true, names are not taken into account while merging adjacent ranges.; If you are not using the range names for anything, you should set aggressive to true in order to get the lowest possible memory usage.

iterator begin(): Gets an iterator to the first range in the list.

iterator end(): Gets an iterator one past the last range in the list.

const_iterator begin() const: Gets a const iterator to the first range in the list.

const_iterator end() const: Gets a const iterator one past the last range in the list.

size_type size() const: Gets the amount of ranges in the list.

void clear(): Clears the list of ranges.

Static lists

class static_list is optimized for minimal memory usage and high locality. If you need to test IPs against a list, this should be used over the normal list class. It is possible to construct a static_list using a list.

class static_list : boost::noncopyable {
public:
   struct range {
      const wchar_t *name;
      unsigned int start;
      unsigned int end;
   };

   static_list(const p2p::list &l);

   int size() const;
   unsigned int ip_count() const;

   const range_type& operator[](int index) const;

   const range_type *operator()(unsigned int ip) const;
   const range_type *operator()(const range &r) const;
};

static_list(const list &l): Constructs an optimized static_list from an existing list.

int size() const: Gets the amount of ranges in the static_list.

unsigned int ip_count() const: Gets the amount of IPs in the static_list.

const range& operator[](int index) const: Gets the range at an index into the static_list.

const range* operator()(unsigned int ip) const: Returns the range which contains an IP, or NULL if no range is found.

const range* operator()(const range &r) const: Returns the range which collides with another range, or NULL if no range is found.

Compact lists

class compact_list is used as a lightweight optimized list to test IPs and ranges against. It is akin to static_list but does not store range names, achieving the lowest possible memory usage and best locality. It is possible to construct a compact_list using a list.

#include <utility>
#include <boost/utility.hpp>
#include "p2p/list.hpp"

class compact_list : boost::noncopyable {
public:
   typedef std::pair<unsigned int,unsigned int> range_type;

   compact_list(const list &l);

   int size() const;
   unsigned int ip_count() const;

   const range_type& operator[](int index) const;

   const range_type *operator()(unsigned int ip) const;
   const range_type *operator()(const range_type &r) const;
   const range_type *operator()(const range &r) const;
};

compact_list(const list &l): Constructs an optimized compact_list from an existing list.

int size() const: Gets the amount of ranges in the compact_list.

unsigned int ip_count() const: Gets the amount of IPs in the compact_list.

const range_type& operator[](int index) const: Gets the range at an index into the compact_list.

const range_type *operator()(unsigned int ip) const: Returns the range which contains an IP, or NULL if no range is found.

const range_type *operator()(const range_type &r) const: Returns the range which collides with another range, or NULL if no range is found.

const range_type *operator()(const range &r) const: Returns the range which collides with a p2p::range, or NULL if no range is found.

Parsing/Saving lists

libp2p parses blocks of memory directly. This allows you to efficiently memory map files. When saving lists, libp2p uses an ostream.

Basic parsing

#include "p2p/list.hpp"

void load_p2p(list &l, const char *data, const char *end);
void load_p2b(list &l, const char *data, const char *end);
void load_dat(list &l, const char *data, const char *end);

void load_p2p(list &l, const char *data, const char *end): Loads a P2P list.; If the file does not begin with a UTF-8 BOM, it assumes the file is encoded in ISO-8859-1.; Throws utf8_error if the list contains an invalid UTF-8 sequence.

void load_p2b(list &l, const char *data, const char *end): Loads a P2B list.; Throws p2p_error if the list is corrupt or doesn't appear to be of the P2B format.; Throws utf8_error if the list contains an invalid UTF-8 sequence.

void load_dat(list &l, const char *data, const char *end): Loads an eMule .dat list.; If the file does not begin with a UTF-8 BOM, it assumes the file is encoded in ISO-8859-1.; Throws utf8_error if the list contains an invalid UTF-8 sequence.

Saving

#include <ostream>
#include "p2p/list.hpp"

void save_p2p(const list &l, std::ostream &os, bool utf8 = false);
void save_p2b(const list &l, std::ostream &os);
void save_p2b1(const list &l, std::ostream &os);
void save_p2b2(const list &l, std::ostream &os);
void save_p2b3(const list &l, std::ostream &os);
void save_dat(const list &l, std::ostream &os, unsigned int access, bool utf8 = false);

void save_p2p(const list &l, std::ostream &os, bool utf8 = false): Write a given list to an ostream using the P2P format.; If utf8 is true, a UTF-8 BOM will be written before the list and range names will be encoded in UTF-8.; If utf8 is false, range names will be truncated to ISO-8859-1.

void save_p2b(const list &l, std::ostream &os): Write a given list to an ostream using the latest P2B format.

void save_p2b1(const list &l, std::ostream &os): Write a given list to an ostream using the P2Bv1 format.; Warning: wide character range names will be truncated to ISO-8859-1. This function is included for compatibility reasons only; it is strongly recommended that you use save_p2b2 instead.

void save_p2b2(const list &l, std::ostream &os): Write a given list to an ostream using the P2Bv2 format.

void save_p2b3(const list &l, std::ostream &os): Write a given list to an ostream using the P2Bv3 format.

void save_dat(const list &l, std::ostream &os, unsigned int access, bool utf8 = false): Write a given list to an ostream using the eMule .dat format.; access should be >=128 if this is an allow list, or <=127 if this is a block list.; If utf8 is true, a UTF-8 BOM will be written before the list and range names will be encoded in UTF-8.; If utf8 is false, range names will be truncated to ISO-8859-1.

Utility functions

#include "p2p/list.hpp"

enum list_type {
   format_unknown = -1,
   format_p2p     =  0,
   format_p2b     =  1,
   format_dat     =  2
};

list_type determine_format(const char *data, const char *end);
void load_list(list &l, const char *data, const char *end);

list_type determine_format(const char *data, const char *end): Attempts to determine the format of a list.

void load_list(list &l, const char *data, const char *end): Loads a list. If it is unable to determine the format, throws a p2p_error.

Exceptions

#include <exception>

namespace p2p {
   class p2p_error : public std::exception {
   public:
      p2p_error(const char *msg);
      const char *what() const;
   };
}

namespace utf8 {
   class utf8_error : public std::exception {
   public:
      utf8_error(const char *msg);
      const char *what() const;
   };
}

class p2p_error: p2p_error is thrown if a corrupt list is passed to load_p2b or if a list of unknown format is passed to load_list.

class utf8_error: utf8_error is thrown if an invalid UTF-8 sequence is encountered while loading a list.

Example

This example efficiently loads lists of any format, merges them, optimizes them, and prints the result. We take advantage of multiple CPU cores (if available) using OpenMP.

#include <iostream>
#include <exception>
#include <p2p/list.hpp>
#include <p2p/format.hpp>
#include <p2p/utility.hpp>
#include <omp.h>
#include <tchar.h>
#include <windows.h>

static bool LoadList(p2p::list &list, LPCTSTR file) {
   HANDLE fp = CreateFile(file, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
   if(fp == INVALID_HANDLE_VALUE) return false;

   DWORD len = GetFileSize(fp, NULL);
   if(len == INVALID_FILE_SIZE) {
      CloseHandle(fp);
      return false;
   }

   HANDLE map = CreateFileMapping(fp, NULL, PAGE_READONLY, 0, 0, NULL);
   if(map == NULL) {
      CloseHandle(fp);
      return false;
   }

   void *view = MapViewOfFile(map, FILE_MAP_READ, 0, 0, 0);
   if(!view) {
      CloseHandle(map);
      CloseHandle(fp);
      return false;
   }

   bool ok;

   try {
      p2p::load_list(list, (const char*)view, ((const char*)view) + len);
      ok = true;
   }
   catch(std::exception&) {
      ok = false;
   }

   UnmapViewOfFile(view);
   CloseHandle(map);
   CloseHandle(fp);

   return ok;
}

int _tmain(int argc, TCHAR *argv[]) {
   p2p::list list;
   int i;

   #pragma omp parallel for
   for(i = 1; i < argc; ++i) {
      p2p::list tmp;
      LoadList(tmp, argv[i]);

      #pragma omp critical
      {
         list.splice(tmp);
      }
   }

   list.optimize();

   p2p::save_p2p(list, std::cout);

   return 0;
}

Open Source

libp2p is OSI Certified Open Source Software under a zlib/libpng license, which makes it easy to integrate into just about any applications that wants to use PeerGuardian lists.

External links

Official PeerGuardian website

Retrieved from "http://wiki.phoenixlabs.org/wiki/Libp2p"

Libp2p

From Phoenix Labs Wiki

Contents

Reference

Ranges

Lists

Static lists

Compact lists

Parsing/Saving lists

Basic parsing

Saving

Utility functions

Exceptions

Example

Open Source

See also

External links

Views

Personal tools

Navigation

Search

Toolbox