yPass.net

Data Import

With the mongofiles multiput patch:

$ time find . -type f | sed 's/^\.\///' | /tmp/mongofiles -d test multiput -
real    199m34.581s
user    2m25.931s
sys     4m36.540s

Without the patch:

$ time find . -type f -exec mongofiles -d test put {} \; 
real    598m40.087s
user    103m36.993s
sys     102m13.679s

GridFS vs. ZFS - Concurrency 50 - Requests/Second

As expected, GridFS serves data much slower than directly from the filesystem. Unless you have a massive amount of static data, you should not store the files on GridFS.

GridFS - Concurrency 50 - Requests/Second

Serving files from GridFS that are 50k or larger will saturate a gigabit Ethernet connection.

ZFS - Concurrency 50 - Requests/Second

Serving files direct from disk that are 25k or larger will saturate a gigabit Ethernet connection. There is a serious performace drop seen with serving files over the network that are 10k and larger, however, the network utilization at this point is only 451Mbits/s. Network utilization peaks with 50k files at 913Mbits/sec