Torrust scrape has packed data errors and is not properly formatted

[dev1z]

Hello, there seems to be an error with packed data when scraping announce both udp and http/s.

The format should be d5:filesd20, but on gbitt and torrust it is random, sometimes it is d5:filesd50, d5:filesd54, etc. Thats why almost all scrapers returns invalid data for info hash.

Seems like it is using aquatic source for scraping, but aquatic returns correct bytes of 20, where torrust doesn't.

@Power2All @WarmBeer

[mickvandijke]

Thank you for the heads-up. I've implemented a fix for this in the coming release: aac9ac7

[dev1z]

Thank you @WarmBeer. I have 2 questsions:

1# Will this implementation also work for udp scrape?

2# Any chance to have an explanation about peer_timeout config variable. Is this deleting peers after 900 seconds if the peer returns event=stopped?

[mickvandijke]

1# No this was a fix for HTTP only, I'm looking into UDP atm.

2# The peer_timeout variable is the max amount of seconds a peer may be inactive (after last announce) before it is removed by the tracker. This is for peers that stopped announcing, but never sent a stopped event. Peers that send a stopped event get removed instantly.

[dev1z]

1# Great one, being able to use udp to scrape is also needed as most public trackers need to have it.

2# I understand. What is the best way to use torrust as private tracker to track user ratio? Is there some way to insert into database peers via api or something (that would also need to import event=stopped peers as well)

[Power2All]

I let @WarmBeer look into this, but UDP works fine as far as I'm aware on the development branch.
Right now, I'm making a total rewrite of the core by switching from WARP to the faster and better performing ACTIX framework.
What is exactly the problem with the UDP scraping if I may ask, you got some wireshark or debugging information on which we can debug it on ?

[dev1z]

Seems like I had some issues when testing on windows. Works fine when scraping udp on ubuntu.

Not an issue with tracker.

[dev1z]

@WarmBeer Ok, this definitely is not fixed.
I just found out udp scraping works on < 2.2.0 version, everything >= 2.2.0 returns invalid scraping response on udp.

Tested locally and scraped with https://github.com/medariox/scrapeer and https://github.com/Novik/ruTorrent

[mickvandijke]

I've tested the UDP scraping with https://pypi.org/project/tracker-scraper/ and the result was valid and as expected, so I'm not sure why it doesn't work in your case. Are you sure UDP is enabled on the tracker and does it run in Public mode?

[josecelano]

@WarmBeer Ok, this definitely is not fixed. I just found out udp scraping works on < 2.2.0 version, everything >= 2.2.0 returns invalid scraping response on udp.

Tested locally and scraped with https://github.com/medariox/scrapeer and https://github.com/Novik/ruTorrent

Hi @dev1z, Instead of using an external scraper, I'm trying to reproduce the error with an integration test in this PR.

It would be nice if you could tell me the UDP request and response. And why the response is invalid.

We could also change the workflow to run tests on different OSs in case your error happens only on Windows.

[Power2All]

Could you tell me if Gbitt still shows UDP scrape errors ?
It's build from scratch based on Torrust-Tracker, but if that works fine, I could look into and find the possible issue, perhaps fix it as well.