Skip to content

Scrape multiple torrents simultaneously #106

@stephen304

Description

@stephen304

The UDP tracker protocol allows multiple torrents to be scraped in the same request, up to around 74 according to bep15 (although I've gotten close to 100 on certain trackers.)

Being able to scrape so many torrents in a single request is a huge time saver when doing bulk actions. I'd like to request the ability to initialize the client with an array of torrent, to allow scraping a whole array of torrents at the same time.

This, of course, would have certain limitations. You would probably want to restrict the client from using anything other than scrape (plus it would be extra work anyway). Additionally, there's the matter of the trackers. Ideally, we would want to parse the list of trackers and see which torrents have the same tracker so they can be done at the same time, but since that would require more work, it would probably be sufficient to just add a parameter to .scrape() to pass in 1 or more tracker to query for all torrents in the client. Also, http trackers would obviously be ignored since bulk scrape can't happen with those.

All the scrape events could just reuse the existing .on('scrape') callback since the callback gets the torrent info hash, so a user of the library would be able to tell which tracker and torrent the scrape results are for.

I hope you will consider this a possibility - my torrent indexing project requires lots of tracker scraping, and I would rather avoid making it look like I'm ddosing trackers with udp packets.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions