Transferring Data with BBCP

Rsync is a quite known utility if we want synchronize data between two nodes or even between two folders in the same system, and of course also used for backup purposes. Its main peculiarity is that it sends only incremental changes; for instance, if you get a connection drop when using classic FTP you typically have to re-send the entire folder or file, but with this tool you re-run it again and it will start from where it left.

However Rsync is not so good if you want speed and it doesn`t have any multithreaded capability; so if you have GBs or even 1 TB of data to transfer it can be quite slow in the end.
To overcome these drawbacks you can use BBCP, which is a point-to-point network utility written by Andy Hanushevsky at  SLAC with the goal to get close to the line speed in a LAN or WAN connection.

You don’t need any server listening or any SSH daemon, but you have to make sure bbcp is installed and in the PATH in BOTH systems.

Assuming GIT is installed, below are the steps to install it:

$ git clone


$ cd bbcp/src
$ make

The pre-requisites are:

GNU C++ compiler
Zlib library
Pthreads library


Some examples of usage are:

bbcp -P 2 -V -w 8m -s 16 /local/path/bigfile.tar remotesystem:/remote/path/bigfile.tar

-V verbose output
-P 2 display progress every two seconds
-s 16 create 16 parallel network streams (or threads)
-w sets to 8 MB the size of the disk input/output (I/O) buffers


To transfer a directory just use the -r option (it stands for recursive of course).

bbcp -r -P 2 -V -w 8m -s 16 /local/path/* remotesystem:/remote/path

To resume files in case of a lost connection add the -a and -k switch.

bbcp -r -k -a -P 2 -V -w 8m -s 16 /local/path/* remotesystem:/remote/path

If a firewall is blocking the communication between source and destination, use the -z option

bbcp -P 2 -V -w 8m -a -k -z -s 16 /local/path/bigfile.tar remotesystem:/remote/path/bigfile.tar


I have tested BBCP only on Linux, but it should be possible to compile it and install it in all major UNIX systems. The Windows platform  is not supported.