nav search
Data Center Software Security Transformation DevOps Business Personal Tech Science Emergent Tech Bootnotes BOFH

Fujitsu shrinks SMB file transfer metadata traffic jams

WAN optimiser weeds out extraneous duplication – and everyone's a winner

By Chris Mellor, 22 Jun 2015

Fujitsu has found a way round repetitive metadata ops that delay CIFS and SMB transfers from remote file-sharing sites. It's developing a WAN optimiser product using this software tech.

File metadata is sent too many times in a transfer session, slowing data transfer. What Fujitsu engineers found was that, with a large number of files in a folder, all of the file-attribute information is requested for each file. The accumulation of these requests in the remote network access session causes significant latency.

At the other end of the spectrum, when sending relatively large files, their data is split into pieces tens of kilobytes in size, with header information attached to each segment. Because this header information is updated each time, the transmitted data becomes different even if it sends the same file, which makes deduplication ineffective. That is a blow.

By having proxy software agents at each end of the link, Fujitsu's engineers reduced the effect of these two problems with CIFS (SMB 1.0) and other SMB protocol use. A read-ahead proxy prepares the read-ahead of multiple files' metadata and sends them across the network to the client-side proxy, which uses that data to respond to information on each component file's attributes.


Overview of Fujitsu's remote file transfer acceleration scheme

The Fujitsu engineers also looked at the large file transfer problem and separated each segment into its header and the data package, meaning that the data opackets could be deduplicated, thus reducing the amount of data sent over the link.

Sending a bundle of 100 1KB files in a folder was accelerated ten-fold, while a large file transfer – previously slowed up because dedupe didn't work – was accelerated up to twenty times when headers were separated from the data package in each segment.

Fujitsu says its concept is software-only and can be installed on existing file-sharing systems and applied to cloud and server-virtualisation environments, as well as mobile device remote data access across a variety of network services.

It plans to implement this tech into a product to provide a WAN optimisation capability some time before March 2016. ®

The Register - Independent news and views for the tech community. Part of Situation Publishing