Help with scraping lists

**marcolav** · 06-09-2010

Not sure if this is the right thread, hope it is !

Can somebody assist me in using scrapebox to generate lists

Ok, I know how to scrape a list of vbulletin sites, sort them by pagerank etc, now, my questions are

1) will any URL do and SickSubmitter will figure the registration page? Or do I have to import the registration URL itself?

2) I think I saw a post somewhere explaining how to check if the forum allows for public profiles, but I cannot locate it now and I remember I could not really understand it so I would be grateful if somebody could take some time to explain it to a technically challenged person like myself... ;o)

thanks in advance

Marco

**NiyazK** · 06-09-2010

this might help you
Packet Making Tips - Sick Marketing Forums

**marcolav** · 06-09-2010

Originally Posted by NiyazK

this might help you
Packet Making Tips - Sick Marketing Forums

Thanks, that was helpful indeed, still got some questions...

1) which is the URL I need to import into Sick Submitter? Any Url or must it be the register URL?

Thx

Marco

**1buckplr** · 06-09-2010

Originally Posted by marcolav

Thanks, that was helpful indeed, still got some questions...

1) which is the URL I need to import into Sick Submitter? Any Url or must it be the register URL?

Thx

Marco

Any URL will work as long as it is within the forum (a forum thread or member profile page or forum User List page etc etc)

Basically as long as the register link is somewhere on the page it will work.

**marcolav** · 06-09-2010

Originally Posted by 1buckplr

Any URL will work as long as it is within the forum (a forum thread or member profile page or forum User List page etc etc)

Basically as long as the register link is somewhere on the page it will work.

Actually, that doesn't seem to work, I have just ran a test with 551 Expression Engine url's and although the member url's i scraped do show a 'Register' link, I am getting A TON of

FAIL-registration page not found

Here is a couple of sample URL's that failed

Revolver Forum | theumpteenthtimes
Wizcode | GaryGregory

Any suggestions most welcome!

Marco

**SickMarketing** · 06-09-2010

You may want to follow the steps here for multiple sites failing.

Must Read! Basic Trouble Shooting - Sick Marketing Forums

Should give you an idea of exactly what is happening.

Sick

**1buckplr** · 06-09-2010

Originally Posted by marcolav

Actually, that doesn't seem to work, I have just ran a test with 551 Expression Engine url's and although the member url's i scraped do show a 'Register' link, I am getting A TON of

FAIL-registration page not found

Here is a couple of sample URL's that failed

Revolver Forum | theumpteenthtimes
Wizcode | GaryGregory

Any suggestions most welcome!

Marco

Both those sites work fine for me using standard sick wizard template, no errors so I really dont know whats happening for you.

On a sidenote, to put things into perspective, even if I scrape for Member profiles I will expect 50% failure minimum for various reasons like
- custom fields
- no new members
- etc etc

Even more perspective -
I currently have scraped over 4000 Expression engine forums, of which less than 1500 worked in the wizard with no editing involved

**marcolav** · 06-09-2010

Originally Posted by 1buckplr

Both those sites work fine for me using standard sick wizard template, no errors so I really dont know whats happening for you.

On a sidenote, to put things into perspective, even if I scrape for Member profiles I will expect 50% failure minimum for various reasons like
- custom fields
- no new members
- etc etc

Even more perspective -
I currently have scraped over 4000 Expression engine forums, of which less than 1500 worked in the wizard with no editing involved

Odd, i just pasted those 2 url's by themselves in the list wizard, imported them as a separate list and they worked...

However, both with these and with all the ones that were successfully registered in the previous run, I am now facing the problem that no links are being added. I tried to have the window visible and pause before exit, i can only see Sick Submitter logging in and then zipping through the next window, but I don't see any links being added, and in fact when viewing profiles , there are no links . These are all EE forums, what am I doing wrong here? I am using the standard list wizard template ( I wouldn't even know how to change it anyways) so I am assuming it should work?

marco

**cafi** · 06-09-2010

Hi marcolav,
For better success I Trim my EE links to the root of the forum
In links I will put just

Code:

http://www.revolvermag.com/forum

The EE script do this, but is more safe this way.

Also keep in mind that not all EE allows signatures/Bios link
The page not found, I got them a lot when I scrape urls, many sites have moved to other platform or just don't exist anymore.

About no links. Please verify if you are using advanced links or normal and if you have the fields filled. EE script do this link > [your About me] [url with title] + ... (more links if advanced)

Does the SickSubmitter scripts with EE work well?

**marcolav** · 06-10-2010

Ok, maybe I am not enough tech savvy for this software but I am getting a ridiculously low success rate.

I have imported a list of EE sites using the list wizard, got about 70 registrations out of 551 sites, but that's ok.
However:

1)a lot of the failed ones are showing 'Captcha Error' which I don't understand:
i am using decaptcher, details are entered and clicking test does return the current account balance so details are indeed correct

2) despite the registration successful message, when I go to submit, nothing is submitted no links are added , nothing.

3) is there some edit, some change that needs to be made to imported lists, even if using a script for which a a template does exist? If so, is there a guide, a video, anything that explains what and how?

marco