BitTorrent’s DHT and the Leading ISP Networks Helping to Keep it Alive

Nearly a quarter of a century after its debut, internet users stocking up on the latest multi-GB Linux distros can still do so with help from the BitTorrent protocol.

Able to download chunks of even the largest files, distributed among other users who could be anywhere on the planet, BitTorrent’s file transfer skills are two decades old yet never fail to impress.

For those yet to sample the magic of magnet links, the whole process begins with a humble .torrent file. These relatively small files contain metadata relating to the large file the user wants to download, including the location of one or more ‘trackers’, the central online servers that facilitate communication between users’ torrent clients.

Yet the real magic lies in BitTorrent’s resilience; it has a secret weapon that can find those chunks of data, wherever they may be, even when central trackers are blocked or shut down. It’s called DHT – Distributed Hash Table – and it’s one of the key reasons BitTorrent still performs so well on the global stage, even today.

Trackers Are Useful But Disposable

A central server known as a tracker communicates with torrent clients to coordinate transfers of files between them. When a client requests a piece of a file the tracker knows is available, the tracker tells the client which other clients have it and from there, transfers take place peer-to-peer. Periodically, clients update the tracker with new information about the network, and so the cycle continues.

DHT, on the other hand, empowers torrent clients with the ability to receive and impart information directly with other clients. By creating distributed network knowledge, there’s no absolute need for a tracker, eliminating a potential central point of failure.

Peers knowledge of other peers is boosted by Peer Exchange or PEX, a system through which a client that’s connected to another client, shares the IP addresses of peers it already knows. This increases the recipient client’s pool of connectivity opportunities and, if all goes well, speeds up downloads.

When new clients join the DHT, other torrent clients share content locations with the newcomer and, given time, the client returns the favor by passing information to other clients following similar requests.

The process is boosted locally using PEX, but the aim is the same – discovery of other clients/peers to create a more robust network with more effective file transfers.

In short, DHT makes every client a node in a vast decentralized network, with each sharing their growing knowledge of the network with other clients, that in turn have knowledge they’ll automatically share with others.

No single client knows the location of everything and even the pooled knowledge of thousands is unlikely to produce a full network map. However, most clients know enough to point out the location of at least something useful, and with PEX, they’ll provide the locations of other clients which helps to strengthen the overall network.

To many people the sharing of content is the most visible aspect of BitTorrent, but its real strength lies in the sharing of information that maintains the underlying network. The network that underlies that obviously plays a massive role too.

Tens of Millions of Peers But Hard to Measure

In the bigger picture, the loss of few clients from public DHT is a non-event. Indeed, the sudden shut down of a major residential internet provider somewhere in the world may not be especially disruptive either, given the scale of the network. Yet putting an exact figure on the size of the network has to date proven elusive.

The lowest estimates always start in the millions of peers, with some researchers previously reporting anything from 20 to 30 million, sometimes a few million more, other times a few million less.

Researchers behind BoonTorrent, a now seven-year-old project to create a real-time monitoring tool for BitTorrent DHT traffic, reported that “BitTorrent traffic is abundant, but difficult to analyze. To capture enough data for significant analysis, a large distributed solution is needed.”

Their solution included a heatmap visualization of the previous two minutes of traffic, with two examples shown below.

BoonTorrent DHT Heatmap 1

BoonTorrent DHT Heatmap 2

Depending on the circumstances, we know that the number of peers can vary by at least 10 million, possibly more at times, or less.

That being said, these images suggest that Southeast Asia and Russia had quite the presence on BitTorrent’s DHT seven years ago. That’s a long time in internet years and everything is subject to change.

Some ISPs Play Big Roles Supporting DHT Worldwide

As far as we’re aware, there are no large, recent studies on BitTorrent’s DHT so the overall numbers remain as elusive as ever. However, when using the tools available at IPinfo.io, something unexpected appeared; tracking data for BitTorrent’s DHT and some estimates based on IPinfo’s visibility of the network.

The data displayed as of today concerns the number of IP addresses and their corresponding ASNs observed on the Mainline DHT in the past 30 days. The data obtained by TF from IPinfo.io just a few days ago is broadly the same.

When drilling down into the details, the data provides an overview of the ISPs with the greatest number of allocated IP addresses observed on the DHT network.

Top ISPs With a Strong Customer DHT Presence

With 3,152,801 IP addresses observed on the DHT during the previous 30 days, Russian ISP PJSC Rostelecom comfortably takes the top spot. Lagging two million IP addresses behind, Korea Telecom takes second place (1,143,168 IPs) with CHINANET-BACKBONE narrowly behind with 1,131,734 observed IPs.

Positioned between ISPs from China and Morocco, Comcast Cable Communications takes 9th place with 377,976 observed IP addresses, with the UK’s British Telecommunications PLC edging out competition from Egypt and Turkey with 218,365 IP addresses. Given the site-blocking measures in place at BT, that’s a considerable number of subscribers sharing content openly, with no apparent need for a VPN.

The full list is available below, with a bonus column of most popular content downloaded by users in each country, tracked by IKnowWhatYouDownloaded on the DHT itself. Ultimately, the data provides a clear snapshot of BitTorrent’s modern landscape and the international telecom giants its users connect from.

It’s worth noting that a) this view of the DHT may not be complete and b) countries without a dominant national ISP may have many smaller ISPs not covered in the report. Even with that caveat, pushing Russia from the top spot overall seems unlikely since it already appears three times in the same table.