Ahmedul has a law on CPU design. Simply put, this means that if some process contributes 10% to your performance, optimizing it can’t improve more than 10%. Common sense, really, but it explains the importance of knowing how fast or slow different parts of your system are. So how fast is Linux Pipe? That’s a good question and that’s one [Mazzo] Go out to answer.
The inspiration was a highly-optimized phishing program that clocked in at 36GB / s on his laptop. Is that a normal speed? No. A typical program using pipes on the same machine is not 4 GB / s. What accounts for the difference?
The following is a great use of key performance measuring tools and optimization. There are a lot of revelations about how the pipes work inside the kernel along the way. Also, there are a few more things behind, including paging and huge pages. For example, using huge pages is responsible for 50% improvement.
Has he finally reached 36GB / s? No, he actually got 75 GB / sec! There is a lot to learn in both specific techniques and methods. At each step, careful profiling has identified what is being eaten most of the time and that time has been reduced. The results are self-explanatory.
We’ve talked about piping many times in the Linux Fu installment. You can communicate with the pipe. You can even name them.