Newsgroup download automation

DavPaz · **Joined:** 27th Jun, 2008 **Posts:** 38654

Ah, I have a question.

When sickbeard downloads shows, is it supposed to copy the files into the correct folders? It hasn't done that for the Smallville episodes. Have I missed a step?

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

Possibly 1 step, possible multiple steps!

You need to ensure that sabnzb is set to post process using the sabtosickbeard script for the TV category.
You need to make sure that the autoprocess.cfg file has the correct settings for your sickbeard installation and is named correctly.
There might also be a setting on sickbeard that you need to change too, I forget

If you press the plus sign on the left of a completed download on sabnzb, then there will be a message about postprocessing which will probably be a failure, and a link to the log file which should give you an idea...

DavPaz · **Joined:** 27th Jun, 2008 **Posts:** 38654

It works!

I came home today to find the latest Glee episode downloaded, renamed and in the right folder. This is truly excellent.

Thanks for the help, chaps.

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

I am about to embark upon the Quest For Automagic Yarring.

Wish me luck, I'm about to fiddle with me beards and that. Expect questions.

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

Got SABnzbd, NZBMatrix and Giganews working

Trialled a couple of TV shows, all seems to be working together like the well oiled crew of the Good Ship Yarr and was straightforward enough.

Next up, Sick Beard.

Presumably, if I download the source from the Sick Beard Site I need to then compile it into a Windows .exe to install?

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

Windows, pish, no idea. Thought there was a precompiled exe somewhere or other?

Us Mac user look down our noses at you guys, you're lucky I replied at all...

kalmar · **Posted:** Thu Oct 28, 2010 9:52

It's python, so you just go "python sickbeard" and it runs. Bit fiddly but it seems to be working OK on my mac.

I've been too busy to bother with it since I set it up, but I kicked it off last night and it's downloaded the previous episodes of Lip Service and seemingly random episodes of Chuck in sabnzbd, so that's nice.

DavPaz · **Joined:** 27th Jun, 2008 **Posts:** 38654

I used a pre compiled exe. From that very download link infact.

Cras · **Joined:** 30th Mar, 2008 **Posts:** 49244

No, you just save it and run sickbeard.exe.

If you want it to run as a service so it runs without logging in, there's some more fiddling about to do:

Quote:

1) Downloaded and installed the Windows resource kit from here: http://download.microsoft.com/download/ ... ktools.exe

2) Copied the files srvany.exe and instsrv.exe from the windows resource kit directory to my SickBeard root install directory (in my case C:\Program Files (x86)\SickBeard) instsrv.exe is the tool that allows you to easily create a windows service and srvany.exe is a dummy program that allows you to specify (via a couple of registry keys) your executable of choice to run....in this case, SickBeard.exe.

3) Droped into a command prompt in my SickBeard root install directory and executed the following:

Code:
instsrv SickBeard "c:\Program Files (x86)\SickBeard\srvany.exe"
4) The above creates the service, now you need to tell it which program to run - in my case, SickBeard.exe, I created a windows .reg file to do this, so it automates the process a little. This is the content of that file, which I called sickbeard_parameters.reg:

Code:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SickBeard\Parameters]
"AppDirectory"="C:\\Program Files (x86)\\SickBeard"
"Application"="C:\\Program Files (x86)\\SickBeard\\SickBeard.exe"
"AppParameters"=""
Create a text file of your choice in your SickBeard root install directory (or somewhere for safe keeping), cut and paste the above into it and either execute it from a command prompt, or double click it from windows explorer etc. Follow the on-screen prompts to add the values to the registry.

5) Use the Windows Services Manager (start>run>services.msc) to set the new "SickBeard" service to start automatically. I personally have set the recovery options to restart the service at every available opportunity - do this via the "recovery" tab.

6) Start the service using the above tool

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

DavPaz wrote:

I used a pre compiled exe. From that very download link infact.

Yes, you're very right.

I am in fact a knob.

Needed to run as administrator, otherwise it throws up an error.

Carry on.

(Also, thanks for replying chaps, even you non Windows scorn-y types

)

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32624

My Sickbeard/etc setup runs on Windows. Scorn away!

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

Thanks for the other stuff too Craster. Will peruse.

Cras · **Joined:** 30th Mar, 2008 **Posts:** 49244

The Egg wrote:

Thanks for the other stuff too Craster. Will peruse.

It looks complicated, but it's actually incredibly simple. It's also something they should have put in the installer, but there you go.

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

Have made progress, have got Sickbeard running and fed it all of the details I think it needs.

I've added a show and am waiting for it to go off and find the 5 or so episodes that I'm missing. It's not done much yet though, even though I know that the episodes are on NZBMatrix.

I'll need to do some more tinkering when I have some more time I think. I'm not sure I've got Sickbeard and sabnzbd talking to each other properly.

However, it does look like a most excellent means of grabbing TV Shows, am very happy so far.

LaceSensor · **Joined:** 2nd Apr, 2008 **Posts:** 3137

maybe not the right place for this, but I use utorrent and recently ive been getting sweet FA download speed.
I havent the skills or time to properly troubleshoot, but i did do some google link to check if bittorrents are being throttled by my ISP and it said it wasnt.
so im a bit confused

and sad that I cant watch tv shows.

kalmar · **Posted:** Thu Oct 28, 2010 15:41

Which ISP is it?
Most of them *do* throttle p2p stuff, whether they are up front about it or not.

On the (budget) tariff I'm on with Plusnet, p2p is basically "off" until midnight-7AM, when it's still quite throttled, you might get one episode of something overnight if you were lucky (300MB).

However it can't recognise newsgroups over SSL as "leeching" so it's not too badly restricted, I can pull down about 2GB per night.

LaceSensor · **Joined:** 2nd Apr, 2008 **Posts:** 3137

o2

never used to have a problem any time of day and its not like ive done any hammering of it in recent days. seems in the last week I can get sweet FA out of my downloads. :/

Hmm

just tried grabbing a file currently queued in Utorrent, using free download manager, and im getting 800k/s, compared to 1k/s in Ut.

wierd.

Wullie · **Joined:** 27th Jun, 2008 **Posts:** 6183

I used to have bother with torrents & my ISP don't do shaping. The downloads often sped up dramatically as soon as I connected to a certain seeder & then slowed as soon as I'd received that chunk. Then back up as I connected for another chunk. etc. etc.

That, coupled with the fact that actual Linux distros

d/l consistently fast leads me to believe that most BT users are either throttled to fuck by their ISPs or their cuntishness.

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

I've had a consistent 320kbs all day with PlusNet, pulling stuff down from News Groups, I'd never, ever get that with torrents.

I've managed to get Sick Beard and SABnzdb talking now - I'd used duff port information - all seems to be working okay.

I'm not entirely convinced that I've got Sick Beard managing and sorting my files properly, I need a bit of a sort out and re-jig of files and folders anyway so with a bit more tinkering I'm confident I'll get there.

If anyone else is thinking of doing this, I am a belm and I've managed to bluff my way through it with the help of the guides and a fair bit of Googling, so give it a try. Now I have all of Boardwalk Empire to enjoy, I guess the real test will be on Sunday night when the next episode is aired, will it automagically grab the episode? We'll see.

LaceSensor · **Joined:** 2nd Apr, 2008 **Posts:** 3137

Wullie wrote:

I used to have bother with torrents & my ISP don't do shaping. The downloads often sped up dramatically as soon as I connected to a certain seeder & then slowed as soon as I'd received that chunk. Then back up as I connected for another chunk. etc. etc.

That, coupled with the fact that actual Linux distros

d/l consistently fast leads me to believe that most BT users are either throttled to fudge by their ISPs or their cat.

Hmm but same file, 2 different clients, wildly different speeds.
that and multiple different downloads on uTorrrent crawling makes me suspect something is up with it.

Wullie · **Joined:** 27th Jun, 2008 **Posts:** 6183

The swear filter's in fine form today, I had to read back to see what the hell I was on about

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

Post processing failed when trying to copy over An Idiot Abroad to my NAS just now, yet worked fine last night in the wee hours when copying across Nikita. Can't see any reason for it in the slightest, hopefully it was a blip and not the beginning of the end for my sickbeard house of cards configuration

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32624

Oops. I've run out of Giganews quota a week early.

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

Yup, my decision to go for the unlimited usenet account seems to have paid off, I thought I would get through about 100gb a month with the shows I download in HD, so I was considering getting a 1Tb account which I hoped would last out the year.

Current download total this month... 300gb!

Sickbeard makes catching up on old show seasons way too easy.

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

I also upgraded my Giganews account this morning - 10gb in a day yesterday. My ISP is going to have a fit. Ah well!

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

Anybody know what I can do about this error, basically all files downloaded but it seems to have ballsed up when unpacking. It's been stuck like that for about 24 hours now.

I think I've broken it

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

It shouldn't be stuck? When i've had that before it just means some of the files you downloaded were corrupt, so you'll need to grab it from somewhere else...

Try restarting sabnzb?

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

Yeah, I gave that a go and it didn;t work.

Looks like it's having a problem with par2, so i need to unrar it myself

Quote:

FAQ

SABnzbd is stuck repairing a file, how do I fix it?

SABnzbd processes files sequentially, so if Par2 stops working the rest of your downloaded files will get hung up. To fix this, open your operating system's task manager and manually kill the par2 process. The item it was working on will be marked as "failed", so you'll have to manually verify and unrar that job.

Ramsea · **Joined:** 20th Apr, 2008 **Posts:** 963

ok, instead of stripping wallpaper, much to the annoyance of the other half, I tried to install Sickbeard with Python.

After a couple of hours I gave up and installed it using windows. After another few hours trying to get it all configured with NZBMatrix and SABNZBD I finally managed to get them to kind of talk to each other when I realised I had put in the wrong API key in one of the fields (I didnt realise there were 2 of them - school boy error!).

Anyway, as I dont know much about this sort of stuff, I'm now quite proud of myself that I can now get an episode which I had missed, from tinternets and it kind of works.

The main problem I have, which is probably really simple is, when I d/l a file I do not know how to auto update (post process) it. It just goes in my specified folder with no change to file name. I have tried copying the contents from autoProcess TV to a newly created script folder etc but I am unsure if I have done this correctly.

I have tried using the manual post-processing which works in so far that it updates my files to show their correct names in their correct folders, but in doing so, it moves the file from my preferred TV show location to the Sickbeard folder?

I'm confused so any suggestions will be much appreciated.

ps. When changing skipped to wanted, it can take sometime to find the info, whereas if I click on the search link in the same place, it finds it almost instantly. Is this normal or have I missed a step?

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

These are the exact things that I am struggling with, the tidying up of the files and post processing. So I'm afraid I'm no help.

I am planning on tinkering with it some more tomorrow so I may stumble upon the answer then.

But secretly I'm hoping a know it all turns up first!

DavPaz · **Joined:** 27th Jun, 2008 **Posts:** 38654

Here's what I did...

In sickbeard > config > Postprocessing, I pointed at my download directory and clicked 'Scan and Process TV Download Directory'

HTH HAND

The Egg · **Joined:** 16th Aug, 2008 **Posts:** 412

The Egg wrote:

Anybody know what I can do about this error, basically all files downloaded but it seems to have ballsed up when unpacking. It's been stuck like that for about 24 hours now.

I think I've broken it

Just in case anyone else comes across this, Sabnzdb didn't work again until I had deleted the broken file. It would connect for about 30 seconds or so and then slowly the speed would reduce to zero. Deleted the broken files from Sabnzdb and it behaved itself again.

I'm planning on manual unpacking the files so that the download wasn't completely lost.

Although my new HTPC has just arrived so plan on getting that up and running for the rest of today and then setting up Sick Beard/sabnzdb on that so that I can leave it to do its thing quietly.

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

DavPaz wrote:

Here's what I did...

In sickbeard > config > Postprocessing, I pointed at my download directory and clicked 'Scan and Process TV Download Directory'

HTH HAND

I think if you use the sabtosickbeard post processing scripts you shouldn't check this option! No idea why though

On my install I have both sickbeard and sab pointed at each other in the various login detail places, have my downloads going in to the TV category on Sickbeard, and on the categories page of sab I have the TV category set to use the sabtosickbeard post processing script.

Make sure the autoprocess.cfg file has the correct sickbeard login settings.

Also, how you added your shows to sickbeard makes a difference, if you added them by searching already created directories, it will move your shows to those directories in the post process. If you added the shows manually, sickbeard created its own directory to move them too...

There are loads of options in both systems that need to be set correctly, if people are still having problems later, I can go through and tell you what each relevant option is and what I have set them to

DavPaz · **Joined:** 27th Jun, 2008 **Posts:** 38654

So, yeah, I've had to upgrde to the unlimited giganews account, oops!

But! I seem to be downloading TV shows twice. Why is doing that?

Any clues?

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32624

When you say it's doing it twice - do you get two sets of output files, or can you see it twice in sabnzbd's queue?

DavPaz · **Joined:** 27th Jun, 2008 **Posts:** 38654

When Sick Beard snatches a file and puts the NZB in the proper place, within a few seconds, it'll do it again. It's done it on pretty much all of the automatic ones.

Google has revealed an 'ignore duplicates' setting in SABnzbd+ that I'll try but it does seem to Sick Beard not talking to itself.

kalmar · **Posted:** Fri Nov 05, 2010 12:50

Mine seems to be working fine still!

However: in the sickbeard main screen, where it has the list of shows, it will have something like 0/12 downloaded for each one - even though it's downloaded several episodes. Not that it matters, just wondered if it works for anyone else.

Cras · **Joined:** 30th Mar, 2008 **Posts:** 49244

I get the same.

kalmar · **Posted:** Tue Jan 10, 2012 10:12

I'm having problems obtaining anything other than quite recent TV episodes.
For example, the first episodes of Weeds won't DL properly - "x repair blocks missing" it'll say, and FAILED. Similarly for any, uh, "feature length file" I've tried to download, it fails in the same way.
Is the problem likely to be the newsgroup provider or the thing where the NZBs come from? Or SabNZBD itself? I use blocknews and NZBMatrix.
PLS HLPS:>

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

It'll be your provider most likely, or it could be the rars are missing the par files.

Give me a nzbmatrix link to a specific episode and i'll give it a go here and see what happens if you like

I uses usenetnow as my provider.

DavPaz · **Joined:** 27th Jun, 2008 **Posts:** 38654

Your provider has a retention period, based on how much you pay for your subscription. I'd check that first.

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32624

Sounds like your provider is ass.

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32624

Trooper wrote:

I uses usenetnow as my provider.

Wow, is that really $12/mo for unlimited downloads? I pay Giganews $35/mo for that! How do you find them?

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

Doctor Glyndwr wrote:

Trooper wrote:

I uses usenetnow as my provider.

Wow, is that really $12/mo for unlimited downloads? I pay Giganews $35/mo for that! How do you find them?

It is

I'm actually on a $30 a quarter deal, rather than monthly, but I don't think they do that anymore. I'm averaging 50-100Gb a month at the moment, but have had a run of months at 400Gb+ with no issue.

Pretty great, in the past year i've had a couple of outages, but I have their US server setup and just used that instead for a day and both times that server has been up while the UK server was down. I've now setup Sab so that it uses 30 connections to the UK servers and 20 to the US servers, which is working just dandy and will max out my connection if I want.
I'd say 1 in 100 things I download have missing block issues, but looking online at the comments it's almost always files that everyone has problems with, so I suspect dodgy uploads rather than provider problems.

All in all, great value and 100% recommended

myp · **Posted:** Tue Jan 10, 2012 10:30

I don't understand how you can download 400GB of Linux distro ISOs in one month.

Doctor Glyndwr · **Joined:** 30th Mar, 2008 **Posts:** 32624

Hmmm. If I move away from Giganews, I'll need to sort out an alternate VPN setup for Netflix on the Boxee. Still, though, hmmmm.

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

Oh it appears usenetnow are still doing their offer $30.49 for 3 months unlimited, either as a one off or a rolling contract.

Trooper · **Joined:** 4th Mar, 2010 **Posts:** 22391

myp wrote:

I don't understand how you can download 400GB of Linux distro ISOs in one month.

I have 4Tb of mainly TV shows in 720p format, during the summer I can have 20-30 shows being downloaded each week, at up to 2Gb each these days. That doesn't include the full series that I have missed and am downloading the whole lot. I should probably watch more TV than 4 episodes of the west wing a week, to be honest

Add to that the occasional film, streaming of video content onto my boxee, steam and work.

It's easy to hit 400Gb+

kalmar · **Posted:** Tue Jan 10, 2012 10:38

Cheers guys. I thought my provider was OK in that respect, maybe it's too cheap

Trooper, I'll take you up on that this evening, ta!

myp · **Posted:** Tue Jan 10, 2012 10:47

Trooper wrote:

It's easy to hit 400Gb+

Be Excellent To Each Other

Newsgroup download automation
There's beard in your sick.

Who is online

Be Excellent To Each Other

Newsgroup download automation There's beard in your sick.

Who is online

Newsgroup download automation
There's beard in your sick.