The reason most often used to be for US/Canadian people that transit is temporarily saturated to US. However, that is very unlikely the case anymore.
The likely reason is that the server is doing extremely well seeding, and there remains no BW available for FTP/SFTP. A server hammering against it's link speed, utilizing hundreds of upstream connections will not allow much speed for a single stream. Increase your thread count to 5-8 and you will get somewhat better speed downloading/uploading to the server, but do not go higher than that. Too many threads will cause crashes, or slowdowns.
This is unsolvable, only increasing servers to 1Gbps unmetered, or weakening the transit quality/limiting server total performance by downgrading hardware would solve this. But as transit is working well, and link speed is maxed out, there simply remains no way to get higher FTP/SFTP speeds than increasing thread count. You can help it by temporarily limiting your rTorrent instance upstream BW if you really have to.
