FTP vs rsync on the NS-K330 with SnakeOS 1.2.0

I recently been playing with my new NS-K330 “NAS”  — a tiny device with one 10/100Mbps LAN port and two USB 2.0 ports, intended for home use as a cheap Network Attached Storage unit and file download client. This post describes the network transfer performance I experienced using ftp and rsync (with and without ssh) once the firmware has been upgraded to SnakeOS.

The details

Having upgraded the NS-K330 to SnakeOS, I wanted to see how effective it would be as the target of regular daily or hourly rsync backups from my other home server(s).

For comparison I transferred a large file (80MB gzipped tarball of a debian tree) using FTP, regular rsync (over ssh) and rsync without ssh. I used two different physical media — straight 1Gbps Ethernet (running at 100Mbps on the NS-K330 side), and PowerlineAV (using a  Netgear XAV2001 connected to the NS-K330 side and a D-Link DHP-307AV connected to my home network’s DGS-1008D LAN switch. The data store on my NS-K330 was a 512MB Kingston USB memory stick, mounted as /usb/sda1

Executive summary: FTP is much faster than regular rsync (which uses ssh for authentication and encrypted transfer), and still ~2 times faster than rsync in daemon/server mode (which uses no encryption and weak authentication).

Speed using ftp to download from NS-K330 over PowerlineAV

Enabled ftp daemon on NS-K330 (running SnakeOS 1.2.0), specified /usb/sda1/ftp as ftp directory with only anonymous access.

Put debian.tgz (~80MB) into /usb/sda1/ftp

From my home FreeBSD server (gjagw, 10.1.1.3), I started ftp client and logged into the NS-K330 (at 10.1.1.51).

Retrieving debian.tgz achieved ~4.2MB/sec, or roughly 33Mbit/sec.

gja@gjagw.space4me.com :ftp ftp://10.1.1.51/
Connected to 10.1.1.51.
220 Welcome to SNAKE OS FTP service
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Switching to Binary mode.
ftp> dir
229 Entering Extended Passive Mode (|||31615|).
150 Here comes the directory listing.
-rwxr-xr-x    1 0        0        85792917 Jan 01 06:28 debian.tgz
226 Directory send OK.
ftp> get debian.tgz
local: debian.tgz remote: debian.tgz
229 Entering Extended Passive Mode (|||17132|).
150 Opening BINARY mode data connection for debian.tgz (85792917 bytes).
100% |************************************************************************************| 83782 KB    4.23 MB/s    00:00 ETA
226 File send OK.
85792917 bytes received in 00:19 (4.23 MB/s)
ftp> get debian.tgz
local: debian.tgz remote: debian.tgz
229 Entering Extended Passive Mode (|||48560|).
150 Opening BINARY mode data connection for debian.tgz (85792917 bytes).
100% |************************************************************************************| 83782 KB    4.24 MB/s    00:00 ETA
226 File send OK.
85792917 bytes received in 00:19 (4.24 MB/s)
ftp>

Comparison with RSYNC over SSH over PowerlineAV

For comparison, pulling the same file using rsync (over ssh) is much slower (~548000bytes/sec, ~4.4Mbit/sec)

gja@gjagw.space4me.com :rm debian.tgz
gja@gjagw.space4me.com :rsync --progress root@10.1.1.51:/usb/sda1/ftp/debian.tgz .
root@10.1.1.51's password:
debian.tgz
    85792917 100%  552.36kB/s    0:02:31 (xfer#1, to-check=0/1)

sent 30 bytes  received 85803469 bytes  548265.17 bytes/sec
total size is 85792917  speedup is 1.00
gja@gjagw.space4me.com :

Just in case the PowerlineAV link had dramatically dropped speed during the rsync test, I immediately re-ran the ftp test. Again, speed > 4Mbyte/sec.

gja@gjagw.space4me.com :ftp ftp://10.1.1.51/
Connected to 10.1.1.51.
220 Welcome to SNAKE OS FTP service
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Switching to Binary mode.
ftp> get debian.tgz
local: debian.tgz remote: debian.tgz
229 Entering Extended Passive Mode (|||54192|).
150 Opening BINARY mode data connection for debian.tgz (85792917 bytes).
100% |************************************************************************************| 83782 KB    4.12 MB/s    00:00 ETA
226 File send OK.
85792917 bytes received in 00:19 (4.12 MB/s)
ftp>

Trying rsync/ssh using ‘blowfish’ cipher makes essentially little difference (ssh man page suggests that blowfish is faster than the default ‘3des’)

gja@gjagw.space4me.com :rsync --progress -e 'ssh -c blowfish-cbc' root@10.1.1.51:/usb/sda1/ftp/debian.tgz .
root@10.1.1.51's password:
debian.tgz
    15892480  18%  583.52kB/s    0:01:59
	[..]

Try RSYNC using daemon mode, no encryption over PowerlineAV

As an alternative, I set up a basic /etc/rsyncd.conf

# more /etc/rsyncd.conf
       uid = root
       gid = root

[ftp]
    path = /usb/sda1/ftp
    comment = ftp export area

#

Started rsync daemon on NS-K330:

# rsync --daemon

Then performed an rsync against the daemon:

gja@gjagw.space4me.com :rsync --progress root@10.1.1.51::ftp/debian.tgz tmp
debian.tgz
    85792917 100%    2.31MB/s    0:00:35 (xfer#1, to-check=0/1)

sent 44 bytes  received 85803491 bytes  2350781.78 bytes/sec
total size is 85792917  speedup is 1.00
gja@gjagw.space4me.com :

Nett impact — substantial increase in throughput! rsync without ssh is way faster than rsync over ssh.

(I tried another FTP test a minute later, and again achieved > 4MByte/sec. So plain ftp is still faster in any case.)

NOTE: rsync in daemon mode has VERY weak login authentication capabilities, even if I assume the SnakeOS rsync implements all the standard rsync auth techniques.

FTP and RSYNC (daemon mode) over wired LAN Link

Replacing PowerlineAV with wired link increases the FTP ‘get’ speed a little to around 5.3MByte/sec (~42Mbit/sec). Two trials shown below (a few more trials achieved similar results).

gja@gjagw.space4me.com :ftp ftp://10.1.1.51/debian.tgz
Connected to 10.1.1.51.
220 Welcome to SNAKE OS FTP service
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Switching to Binary mode.
local: debian.tgz remote: debian.tgz
229 Entering Extended Passive Mode (|||35506|).
150 Opening BINARY mode data connection for debian.tgz (85792917 bytes).
100% |**********************************************************************************************************| 83782 KB    5.37 MB/s    00:00 ETA
226 File send OK.
85792917 bytes received in 00:15 (5.37 MB/s)
221 Goodbye.
gja@gjagw.space4me.com :

gja@gjagw.space4me.com :ftp ftp://10.1.1.51/debian.tgz
Connected to 10.1.1.51.
220 Welcome to SNAKE OS FTP service
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Switching to Binary mode.
local: debian.tgz remote: debian.tgz
229 Entering Extended Passive Mode (|||51404|).
150 Opening BINARY mode data connection for debian.tgz (85792917 bytes).
100% |**********************************************************************************************************| 83782 KB    5.22 MB/s    00:00 ETA
226 File send OK.
85792917 bytes received in 00:15 (5.22 MB/s)
221 Goodbye.
gja@gjagw.space4me.com :

Then setup RSYNC daemon on the NS-K330, and retrival from gjagw achieved ~2.2MByte/sec (~17Mbit/sec).

gja@gjagw.space4me.com :rm debian.tgz
gja@gjagw.space4me.com :rsync --progress 10.1.1.51::ftp/debian.tgz .
debian.tgz
    85792917 100%    2.16MB/s    0:00:37 (xfer#1, to-check=0/1)

sent 44 bytes  received 85803491 bytes  2172241.39 bytes/sec
total size is 85792917  speedup is 1.00
gja@gjagw.space4me.com :

So, over 100Mbit/sec ethernet (1Gbps on the gjagw side) we can get ~42Mbit/sec with basic FTP, and about ~16Mbit/sec using rsync w/out underlying ssh encrypted transport.

Conclusion

To be useful as a backup server if more than a few tens of MByte change every day we really need to find a solution that avoids encryption on the actual file transfer phase(s). Perhaps I could use some ssh-based initial login to NS-K330 which launches a very specific rsync daemon mode just for a one-shot rsync, then shuts down the daemon once the rsync is finished, to minimise potential for anyone else making repeated hack attempts at the rsync daemon’s authentication. Something to think about further….

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: