Thursday, April 21, 2011

USB3 Speeds

I read this little post about USB3 speeds that I found very interesting since I am working on designing USB3 hubs.

by Chris Karr, Director, Software/Firmware Engineering of the Branded Products Group at Western Digital Corporation (and previous author of SpeedTools (QuickBench, ZoneBench, etc.),

I read your article and have some feedback about your Key Insight #1. I strongly disagree that USB 3.0 is slower than eSATA, per se. eSATA is limited to 3 Gb/sec whereas USB 3.0 is a 5 Gb/sec interface. They both use 8b/10b encoding so they are both 80% efficient on paper. (Transport protocol overhead further reduces real world data transfer rates and this overhead varies by bus interface type.) We expect USB 3.0 to be able to crack 400 Megabinary bytes per second (MiB/sec) with the right setup. eSATA will likely never crack 275 MiB/sec unless/until they move to 6 Gb.

The limiting factor with USB 3.0 storage devices to date has been the 3 Gb SATA backend of the first generation of USB 3.0 to SATA bridge chips. The 3 Gb SATA interface limits the potential of the 5 Gb Superspeed USB interface. Plus, converting from SATA protocol to USB protocol adds a bit of additional overhead.

That said, there are two basic approaches to actually deliver max speed over USB SuperSpeed with a bridged SATA solution, neither of which, to my knowledge, have yet been done:
1) Put two 3 Gb SATA ports on the back end of the USB bridge and connect two SATA devices which can saturate both of the SATA busses (think two SandForce-based SSD's in a RAID 0).
2) Make a bridge chip with a USB 3.0 front end and a 6 Gb SATA backend and stuff a 500 MiB/sec enterprise-class 6 Gb SSD or RAM drive behind it.

You will need a very powerful PCI bus (aka Gen 2 running off north-bridge, etc.) to get the best numbers, but we believe USB 3.0 can reach at least 400 MiB/sec once the other system bottlenecks are resolved. Of course, with a single HDD all this comparison stuff doesn’t mean much because they just aren’t fast enough to be meaningfully bottlenecked by either SATA or SuperSpeed USB.

Lastly, look for well-designed integrated USB 3.0 host controllers coming in the near future to further boost USB max data transfer rates.

Wednesday, April 6, 2011

Preserving code formatting in blog posts

In my last post I needed to keep the format of the code snippets. I found this very useful tool that will keep the formatting for you and and generate the raw HTML for it:

Vim Visual Selection Tricks

# You can use filters and vim's execution do some powerful things.
# A filter is any standard UNIX program (sort, awk, grep, cut, paste, sed ...) that can read from stdin and write to stdout.
# Vim's execution mode lets you read in or replace text with the output of a program. If you make a visual selection (say,
# using V or C-V), and hit :, your command prompt will immediately be :'<,'>, which means "Apply whatever command follows
# to the lines included in the visual selection." At this point, you can write !foo to replace the text with the output of
# program foo.

# For example, to sort the text by the python column, select it, hit :, and enter !sort -k5. The whole command will look like
:'<,'>!sort -k5
# Running it will produce:

a b cd 1 p
b b cd 2 y
c b cd 3 t
d b cd 4 h

# Results in:

d b cd 4 h
a b cd 1 p
c b cd 3 t
b b cd 2 y

# For more complex tasks, awk is your friend. A command like
:'<,'>!awk '{ print $1, $3, $2, $4, $5 }'

a cd b 1 p
b cd b 2 y
c cd b 3 t
d cd b 4 h

# will flip the second and third columns (but note that inter-column spacing is collapsed).
# To do maths on a column, try something like
:'<,'>!awk '{ sub($4, $4*2+1); print }'

a b cd 3 p
b b cd 5 y
c b cd 7 t
d b cd 9 h

# Could get the same result with perl if there is only one column of numbers:
:'<,'>!perl -pe 's/\d+/$&*2+1/e'

# To flip the visual selection just select the rows and then:

d b cd 4 h
c b cd 3 t
b b cd 2 y
a b cd 1 p

# This pipes the lines through the unix 'reverse cat' program.

These ideas were taken from here:

Sand here: