This is a short blog post about my experience converting the udpt project (a torrent tracker ) to Rust 2018. udpt was a project that was originally written in C/C++ in 2012, in mid-2018 it was ported to Rust, and now updated to Rust 2018 with the modern async/await syntax. The task introduced a few difficulties due to incompatibilities between async crates.

Components Link to heading

udpt is a small server that has a few components:

  1. UDP server - Handles UDP packets and communicates with the tracker in order to send clients a list of peers, and get stats.
  2. API server - An HTTP server that allows admins to add, remove or flag torrents. It isn’t meant to be open to the public.
  3. Tracker - Manages the state of the application and periodically performs maintenance tasks, such as pruning old data and saving the state.

Udp Server Link to heading

In the previous version, udpt spawned a thread for every logical core on the system and entered an infinite loop of accepting packets and responding. There was a problem when converting to tokio, since tokio doesn’t support sending data on a UdpSocket without mutable access, and the UdpSocket that needs to send packets is the same as the server’s socket. There is an open issue on tokio that attempts to solve this issue, but it hasn’t been active for a while. Forking tokio and merging a (pending) pull-request from tokio that was supposed to fix the issue wasn’t enough, since it only made changes to the UdpSocket, but not to UdpSocket’s ReadHalf and WriteHalf which are required to safely send packets from other threads.

Http Server Link to heading

The http server needed to change too, since it needs to access the tracker’s async mutexes. The previous version was based on actix-web which is a great library but would require too much effort to convert after two major updates and it was written in in Rust’s old futures which isn’t very reader or maintainer friendly when comparing with the new syntax. warp proved to be straight-forward and reliable.

Tracker Link to heading

The tracker had many Mutexes that needed to be converted to an async friendly version, this was simple. The problem was to find an async friendly compression library to replace or work with bzip2. The first attempt to tackle this problem was to look at bzip2’s source code since it had a tokio feature, this failed since the tokio version is 0.1, and is in compatible with the current 0.2. After a quick search async-compression showed up, which looked promising. It didn’t work since async-compression’s AsyncRead/AsyncWrite (and more…) traits where from the futures crate, and tokio’s traits where from tokio ☹. Since the traits from both crates where very similar, It wasn’t too difficult to write a bridge that would allow using both libraries together. The bridge code was written generically and can be found here, in case you run in to the same problem.

Update: May 19, 2020 I couldn’t find this before, but tokio-util has a compat feature that allows converting between future’s and tokio’s AsyncRead and AsyncWrite traits. This feature was just recently clearified in tokio-util’s documentation. This made everything so much simpler, all I needed to do was to add the compatibility traits to the scope and call compat or compat_write.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

use tokio::io::AsyncWrite;
use tokio_util::compat::*;
use async_compression::futures::write::BzEncoder;

/* ... */

fn async save_database<W: AsyncWrite + Unpin>(&self, w: W) {
    // `w` is a tokio AsyncWrite, compat_write will allow it to work
    // with a future's AsyncWrite
    let writer = BzEncoder::new(w.compat_write());
    /* ... */
}

Conclusion Link to heading

It’s fun to re-write and modify old or out-dated software just to challenge yourself and learn new things. I hope that this would be helpful to someone in the future.