For quick file transfers between two computers, SCP is a fine program to use. For more complex, large or regular backups, however, the go-to tool is rsync. It is faster, more efficient and can be used in a wide range of situations. For all its conveniences, [Michael Stapelberg] It was felt that it had one major weakness: it was a tool written in C. [Michael] Philosophically opposed to programs written in C, so he was prepared to implement rsync from scratch in Go instead.
[Michael]The decision to tackle this project is a complex one. His ISP recently upgraded his internet connection to 25 Gbit / s, which means his custom router was interrupted on his network. To solve this problem, he moved his router to a PC with several 25 Gbit / s network cards. Now taking full advantage of the theoretically available speed, he began using a tool called GoCrazy, which turned applications written in Go into their own devices. This means that instead of installing a complete Linux distribution (for example, a router) to perform certain tasks, the only thing loaded into the computer is basically the Linux kernel, the Go compiler and library, and then the Go application itself.
To run a new router with hardware capable of supporting this speed and software written only on Go, the last step was to create rsync to support its functions on its network. This means that rsync itself needs to be created from scratch in Go. Once upon a time [Michael] Having done this final work, he sees that the implementation of his rsync is actually much faster than the version built in C, thanks to the modernization available in Go language and the fact that his router is not running all the craft associated with a standard. Linux distribution.
For a software project of this opportunity, we find [Michael]The step-by-step process is worth noting to try to solve any problem between us. Not only that, refactoring a foundation tool like rsync is a work in itself, leave it to speed up the network which most of us already think is too fast. We’re leaving a ton of details on this build so we definitely recommend checking out his speech in the video below.
Thanks [sarinkhan] For the tip!