Announcement

Collapse
No announcement yet.

[SCRIPT] AUTO CROSS-SEED WITH RUTORRENT & AUTODL-IRSSI

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • [SCRIPT] AUTO CROSS-SEED WITH RUTORRENT & AUTODL-IRSSI

    After spending a little while looking I couldn't find any Rutorrent plugins to allow for automatic cross seeding without having to manually download the torrents from the trackers and add them, which is surprising. I wrote a tiny script Ive been using for a few days that cross seeds everything you download via IRSSI to all your trackers (that have the release) and thought I'd share it. Incredibly simple but I'm pretty pleased with it!

    For it to work it requires 'inotifywait', which is part of inotify-tools. It an awesome tool that monitors the kernel for any changes to the filesystem. Much more efficient that a timed loop and is instant. I'm using Ubuntu and it can be installed by
    Code:
    sudo apt-get install inotify-tools
    More info on the github https://github.com/rvoicilas/inotify-tools/wiki


    You'll also need the IRSSI plugin set up and working to get your releases…

    Create a directory for the script where the torrents are to be added by IRSSI, i.e.
    Code:
    [FONT=Helvetica]/home/star_69/cross-seed/import[/FONT]
    then a directory for the script to hold the torrents while the first one of a matching release is being downloaded, i.e.
    Code:
    [FONT=Helvetica]/home/star_69/cross-seed/staging[/FONT]


    create the script by
    Code:
    sudo nano cross-seed.sh
    and paste in the code below, but change the folder paths to the ones you just created:

    Code:
    #!/bin/bash
    #Simple crossseeding script by Star_69 for Torrent-Invites.com
    
    
    #Create a folder where IRSSI plugin puts the torrent files. Will be used to sort
    import="/home/star_69/cross-seed/import/"
    #This folder is used to store the torrents IRSSI has snatched and will be sent to rtorrent after the first one has downloaded
    staging="/home/star_69/cross-seed/staging/"
    #Folder for your autoload or AutoTools folder where torrents are put to automatically be imported by rtorrent
    #If using autotools specify the root folder. You can create identical folders in the import dir above ie ./Movies/1080p/ and it will send them
    #to the matching directories within the autotools folder, ie /home/YOURUSERNAME/AutotoolsWATCH/Movies/1080p/
    rtorrent="/home/star_69/cross-seed/RTWATCH_FOLDER/"
    clear
    echo "MONITORING $import FOR TORRENTS"
    
    
    #inotify waits until the kernal detects a change in the folder. To test without using irssi by copying the torrents manually, use the CREATE event rather than MOVED_TO
    inotifywait -q -m -e MOVED_TO -r "$import" --format "%w%f" |
      while read file
      do
      #name of torrent file
      filename=$(basename "$file")
      #hopefully a .torrent - irssi initially adds it as a .torrent1 but renames it after download - hence MOVED_TO rather than create on the inotify event
      extension="${filename##*.}"
      #name without the extension.. probably now redundant as irssi prepends tracker name
      releasename="${filename%.*}"
      #trimmed release name without the extension or the tracker prepended name
      trimmedrelease="${releasename#*-}"
      #Replaces spaces in release name for dots. Useful as some trackers (ie TL) sometimes have whitespaces in the torrent name
      trimmedrelease="$(echo ${trimmedrelease} | tr " " ".")"
      #checks its a valid torrent file
        if [ "$extension" == "torrent" ]; then
        echo "$(date +"%d-%m-%Y %T") $filename FOUND!!"
        #create directory based on release name
          #checks if directory doesnt exist
          if [ ! -d "$staging$trimmedrelease" ]; then
          #creates folder based on releasename
          mkdir "$staging$trimmedrelease"
          echo "$(date +"%d-%m-%Y %T") NEW RELEASE so created folder $staging$trimmedrelease"
          #moves to rtorrent watch folder
          mv "$file" "$rtorrent${file#$import}"
          echo "$(date +"%d-%m-%Y %T") SENT $filename to $rtorrent${file#$import} for rtorrent to pickup"
          else
          #copies to matched release folder
          mv "$file" "$staging$trimmedrelease/"
          echo "$(date +"%d-%m-%Y %T") EXISTING FOLDER FOUND. SENDING $filename TO $staging$trimmedrelease/"
          fi
        else
        echo "$(date +"%d-%m-%Y %T") $filename IS NOT A VALID TORRENT - IGNORING"
      fi
    done
    set permissions
    Code:
    sudo chmod 777 ./cross-seed.sh


    run it with the same permissions you run IRSSI (i.e. root or user) otherwise it will get stuck moving the torrents. Its best to use screen like rtorrent and IRSSI as you can detach and it runs in the background
    Code:
    [FONT=Helvetica]screen -d -m -S cross-seed ./cross-seed.sh[/FONT]
    This next script is fired on completion of the first torrent to import the other torrents from the same release and remove the folder in the staging area. Create it by
    Code:
    [FONT=Helvetica]sudo nano rtcomplete.sh[/FONT]
    paste this:
    Code:
    #!/bin/bash
    Releasename="$1"
    Releasename="$(echo ${Releasename} | tr " " ".")"
    staging="/home/star_69/cross-seed/staging/"
    rtorrent="/home/star_69/cross-seed/RTWATCH_FOLDER/"
    
    
    while [ -d "$staging$Releasename" ]
    do
      mv "$staging$Releasename"/* "$rtorrent"
      rmdir "$staging$Releasename"
    done
    set permissions
    Code:
    [FONT=Helvetica]sudo chmod 777 ./rtcomplete.sh[/FONT]
    add the following line to your rtorrent.rc (changing the path)
    Code:
    [FONT=Helvetica]
    system.method.set_key = event.download.finished,notify_me,"execute=/home/star_69/cross-seed/rtcomplete.sh,$d.get_name=" [/FONT]
    [FONT=Helvetica]system.method.set_key = event.download.hash_done[/FONT][FONT=Helvetica],notify_me,"execute=/home/star_69/cross-seed/rtcomplete.sh,$d.get_name="[/FONT]


    You will need to restart rtorrent for saves to take effect
    In autodl-irssi Preferences check the boxes 'Download duplicate releases' and 'use unique torrent filenames'. The first option is self explanatory, the second prepends the tracker name to the torrent file so it doesn't overwrite the file as it downloads an identical torrent name from each tracker.



    On your filters, remove the tracker you've specified so its blank. When the filter finds a release you will get multiple torrents. In the Action tab on the filter change the drop down to Save to watch folder and put in the directory of the import directory you created earlier. inotify works recursively so you can have TV / MOVIE / HD / SD folders within the import folder to match your AutoTools folders (if you use it)




    Now you're all set :)

    Say you have 6 trackers and you have a filter for 'Game Of Thrones'. The 'fastest' site has their torrent downloaded first and its instantly sent to rtorrent for download. All subsequent torrents IRSSI downloads for that release (whilst its being downloaded) get moved to a folder based on the release name in the 'staging' folder, i.e. a dir called Game.of.Thrones.S03E04.720p.HDTV.x264-EVOLVE. When the download is complete, all the other torrents which have been snatched for that release get moved to rtorrent's watch folder (or AutoTools dir) and immediately get seeded, and the folder Game.of.Thrones.S03E04.720p.HDTV.x264-EVOLVE is deleted. Should a tracker be REALLY slow and its not announced while the first one is downloading, it'll still be added, but you'll already have it regardless and will cross seed as normal.

    If you wanted to cross seed on a 'faster' site to build your ratio and download on a 'slower' one first, change the delay accordingly on the trackers section in IRSSI.

    This obviously only works on trackers where the files are identical, i.e. not unrar'ed. Sometimes the file sizes differ slightly but it only downloads the few meg difference i.e. proof or sample and seeds as normal. If you have a few sites that do unrar or are worried about downloading multiple releases unnecessarily you can specify in the filter under MATCHED SITES to include certain trackers with a comma separating them, i.e. putting 'SCC, TL, IPT, TBY' etc are all 'scene' trackers and do not unrar. Obviously you'd need to have a play about...

    Its a quick and dirty but works for me. I hope its useful to someone else! Documentation is not really my strong point, so if you get stuck or have any questions, please shout.

    Happy boosting
    Last edited by star_69; April 28, 2013, 07:36 AM. Reason: Lasty found a bug :)

  • #2
    Nice one dude I set this up manually before but this could be added as a plugin or polished and added to install scripts :).


    sigpic
    _.-=' Special thanks to SiniuS for making me VIP and to Gblaze for graph work :)
    '=-._







    Comment


    • #3
      Linux newbie questions:

      Where do the 2 files have to be located? (cross-seed.sh & rtcomplete.sh) I noticed /bin/bash & /bin/sh?
      Without root/sudo, and if my serviceprovider installs the
      inotify-tools, can i do the rest of the actions myself? Or do i need them to do more?

      Comment


      • #4
        Originally posted by lasty View Post
        Linux newbie questions:

        Where do the 2 files have to be located? (cross-seed.sh & rtcomplete.sh) I noticed /bin/bash & /bin/sh?
        Without root/sudo, and if my serviceprovider installs the
        inotify-tools, can i do the rest of the actions myself? Or do i need them to do more?
        You need to chmod (set file access and restriction) multiple directories so after the files are installed you need root access to do so, after that everything can be called upon by any user (depending on what is and isn't allowed in different modes ofcourse).


        sigpic
        _.-=' Special thanks to SiniuS for making me VIP and to Gblaze for graph work :)
        '=-._







        Comment


        • #5
          Thanks for sharing!

          Comment


          • #6
            Hi star_69,

            I setup the script & folders as described in your tutorial. It seems to work ok most of the time, but i have had some occasions where a tracker is a bit slow (and small releases get downloaded very fast), but it doesnt cross seed it anyway as it should.

            When i look in the staging folder there is still a folder for the release, containing the not-processed / cross-seeded releases.

            Example; 2 fast sites worked ok Monsters.vs.Aliens.S01E04.720p.HDTV.x264-BAJSKORV, but after 2 mins the release was uploaded to 2 slower sites and the .torrent files are now in /cross-seed/staging/Monsters.vs.Aliens.S01E04.720p.HDTV.x264-BAJSKORV/ without moving.

            Any way to troubleshoot this? What could be wrong? :p


            edit: is this because there is no downloading torrent anymore to trigger rtcomplete.sh once finished?
            Last edited by lasty; April 27, 2013, 01:32 PM.

            Comment


            • #7
              That's a good question. I've not encountered a problem like hat to be honest. What the rtcomplete script does is on completion it moves the contents of the folder ie all torrents in the Monsters.vs.Aliens.S01E04.720p.HDTV.x264-BAJSKORV to the watch folder and deletes the dir with the rmdir command. If the tracker is slow and doesnt pick it up, it would treat it like a new release as there is no existing folder in the staging area. It moves the torrent accordingly (to watch folder, which is the automatically cross-seeded) and creates a new Monsters.vs.Aliens.S01E04.720p.HDTV.x264-BAJSKORV for any subsequent releases, but again, its deleted if there isn't any once its completed.

              Are the folders identical? I tried to make allowances for whitespaces, so it puts dots in as some trackers do that... actually, as I'm writing this, that could be why. The rtcomplete.sh might not be looking for the folder as the release might have whitespaces that rtorrent is passing through, but the staging dir doesn't...

              I'll check that out now.. Thanks for testing that.. Back in 5mins :)


              ---------- Post added at 09:08 AM ---------- Previous post was at 08:52 AM ----------

              Thats a good spot. Thanks for that. Hopefully this should fix it. Sorry! Schoolboy error :) I had that problem at first on a couple of torrents, thought I'd fixed it but couldn't of bumped into it again during test. Glad it working well, other than that though!

              One line to replace white spaces with dots to hopefully match with the already space replaced dir in staging folder needs to go in the rtcomplete.sh
              Code:
              Releasename="$(echo ${Releasename} | tr " " ".")"
              So it should look like:

              Code:
              #!/bin/bash
              Releasename="$1"
              Releasename="$(echo ${Releasename} | tr " " ".")"
              staging="/home/star_69/cross-seed/staging/"
              rtorrent="/home/star_69/cross-seed/RTWATCH_FOLDER/"
              
              
              mv "$staging$Releasename"/* "$rtorrent"
              rmdir "$staging$Releasename"
              If you could let me know how you get on, I'd be grateful. Thanks!

              Comment


              • #8
                I edited the file as you explained.. Gonna check how it goes.

                One question; if your 1st download goed so fast that the rest is added after completion, then rtorrent automatically does a hashcheck + resume, right? Could it be that the hashcheck+resume doesnt trigger "event.download.finished"?

                Edit;

                Did not fix the problem. I think it is indeed related to the spaces and dots. The first 2 fast sites have dots and the lats 2 have spaces. The last ones got stuck in the staging folder before the edit. Now i also see them mentioned in irssi, but they are not added to rtorrent or stuck in any folder..

                Edit2:

                I just added the torrent from slowsite 1 manually, and after the hashcheck, the torrent for slowsite 2 appeared automatically. I guess it wa stuck in queue or something like that? *confused*

                Edit3:
                Its getting late, i checked the folders with flashfxp and didnt notice folder cache was on. I guess there was a folder in the staging area, but anyway, the files were stuck there. Gooooodnight! :p
                Last edited by lasty; April 27, 2013, 06:37 PM.

                Comment


                • #9
                  Thanks for checking that Lasty. Just got in. I'm in a bit off a mess. Think you might be on to something with the event.download.finished. I'm going to check it in the morning. Really appreciate the testing. Glad it works ... most of the time! Works a treat for me with the trackers I use (as mentioned in the example)..always the way..! Perhaps it needs a trigger on the hash. At least you're getting the boost on the fast ones ;) .. watch this space!

                  ---------- Post added April 28th, 2013 at 02:41 AM ---------- Previous post was April 27th, 2013 at 03:32 PM ----------

                  Morning [MENTION=228360]lasty[/MENTION]! You were on the money with the hash check. I changed the initial post to add an extra event to the rtorrent.rc and also tweaked the rtcomplete. I added a serious delay to one of my trackers to try to replicate your scenario and so far so good

                  Comment


                  • #10
                    Nice! Awesome work on this script star_69. Tanks for doing all this work! <3


                    Edit: I updated my configuration to the latest versions of your script. Gonna let it run for a while and report back how it goes!

                    ---------- Post added April 29th, 2013 at 11:40 AM ---------- Previous post was April 28th, 2013 at 01:06 PM ----------

                    Originally posted by star_69 View Post
                    Thanks for checking that Lasty. Just got in. I'm in a bit off a mess. Think you might be on to something with the event.download.finished. I'm going to check it in the morning. Really appreciate the testing. Glad it works ... most of the time! Works a treat for me with the trackers I use (as mentioned in the example)..always the way..! Perhaps it needs a trigger on the hash. At least you're getting the boost on the fast ones ;) .. watch this space!

                    ---------- Post added April 28th, 2013 at 02:41 AM ---------- Previous post was April 27th, 2013 at 03:32 PM ----------

                    Morning @lasty ! You were on the money with the hash check. I changed the initial post to add an extra event to the rtorrent.rc and also tweaked the rtcomplete. I added a serious delay to one of my trackers to try to replicate your scenario and so far so good
                    I let it run for halfa day, and overall it runs good! However, i still have some releases that are stuck in the /staging folder. Not sure why they arent triggered to move. Also, i have some release folders that are empty (in staging). Perhaps this is caused by -> a new release coming while the previous release is not fully processed?

                    Perhaps it would be an idea to add some checks;
                    - if releasename is not like foldername & timestamp folder - current time > 5mins, then rtcomplete on foldername.
                    - before rmdir, check if folder contains files (in the case that files are saved there the second its released), if so rtcomplete again.

                    etc.

                    I'm not sure how to debug this exactly, i would like to see what went wrong. perhaps we could do a simple log (with on/off option) to check later.

                    Anyway this script is awesome already, just looking for ways to perfect it (sorry i'm not a coder) :p

                    ---------- Post added April 30th, 2013 at 09:14 AM ---------- Previous post was April 29th, 2013 at 11:40 AM ----------

                    Ok so its been running for another day now. Big releases work flawless. Only the small ones have issues some times. I think it might have something to do with stuff happening simutaniously. (like stated above, perhaps files are added as folder is being removed, thats all i can think of :p) Anyways great work!
                    Last edited by lasty; April 28, 2013, 08:20 AM.

                    Comment


                    • #11
                      Hi,

                      Thanks a lot for your script. It is exactly what I've dreamed of :)


                      BUT I got a problem. The script is well active on my side, but it seems it's just not sending the files to rtorrent watch folder.

                      A little screenshot:



                      So there's of course this "ERROR" that I don't get. But the other inputs without error, are from other trackers and the main problem is the same.

                      First I'd like to say that I set up a watch folder only since I installed your script. I actually thought I had too in order to make this work (I don't have to right ?). And I'd definitely prefer to stick with "send to rtorrent" setting if possible.

                      I can see "cop land' and "dark skies" torrent files in my "incoming" folder (/home/user/cross/incoming). Nothing in "staging" folder, and finally nothing has been downloaded.

                      And also:

                      Once Were Warriors 1994 1080p BluRay DTS x264 D-Z0N3 in Movie/1080p/i
                      What is "Movie/1080p/i" ?

                      Well as you can see I'm a bit lost here. And don't really know where to start... If you can help, that'd be great.


                      Thanks !

                      EDIT:

                      Oh wait... I found the issue... Wrong path in script and config... -_-

                      Thanks !
                      Last edited by Zordac; May 8, 2013, 06:56 AM.

                      Comment


                      • #12
                        Hi,

                        One question:

                        Would there be a way to not use the "send to watch folder" option, but instead "send to rtorrent" ?

                        I ask this because with this first option, many options are unavailable such as "labels", "ratio group"....etc

                        Or maybe you think of another way ?

                        Thanks for your help.

                        Comment


                        • #13
                          Thanks for sharing

                          Comment


                          • #14
                            Curious.... I use RSS to grab my .torrent files with RuTorrent. Most of the time, it will grab from 3-4 sites and download at the same time, and then seed. Is there an issue doing it this way? Seems to work most of the time, but sometimes I get :

                            Hash check on download completion found bad chunks, consider using \"safe_sync\".

                            Going to try to search some more on the forum...


                            Found this.... There has to be a more automated way using RSS.... gonna keep searching.

                            http://www.torrent-invites.com/bittorrent/227827-how-cross-seed-scene-releases-seedbox-using-rutorrent-fast-safe-easy.html
                            Last edited by baztard; May 20, 2013, 07:49 PM.

                            Comment


                            • #15
                              Can this be done without root access? Also, I have heard rumors that cross-seeding won't work correctly if the piece size is the same.

                              Comment

                              Working...
                              X