• You are not logged in.

#1 April 22, 2014 14:34:29

matsrom
Registered: 2014-04-21
Posts: 8
Reputation: +  0  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

Hi!
I am Spanish so I can not explain very well but I will try:

I do not know how to return to the previous page after using SELECT GROUP to capture the content. And how to do that automatically turn to page 2, 3, 4 … 10 and click “…” to go to pages 11-20.

(What I need to do is enter the links on the first page and capture the content, then go to the second page and do the same. And so to all pages.
I'm new and I'm really stuck. thank you very much)

This is the web page:
http://www.brcdirectory.com/Siteresults.aspx?CountryId=0&StandardId=972f3b26-5fbd-4f2c-9159-9a50a15a9dde&

Attachments:
attachment p.fmpx (24.2 KB)

Offline

#2 April 22, 2014 18:32:17

admin
Registered: 2012-03-15
Posts: 289
Reputation: +  1  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

Please see point 2 of How to scrape pages without “next link” here: http://www.fminer.com/faq/, they are the same problems.

Offline

#3 April 23, 2014 10:12:12

matsrom
Registered: 2014-04-21
Posts: 8
Reputation: +  0  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

I have seen that and I tried to do the same but It doesn't work. I don't know what I'm doing wrong and because of it I'm asking form your help.
Thank you.

Offline

#4 April 23, 2014 10:32:23

admin
Registered: 2012-03-15
Posts: 289
Reputation: +  1  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

Made the demo for you, please notice, the links on the site is not real links, they are “button” for ajax. You should not sue “openlink(s)” action, should use “click”, and make a loop. Please watch the tutorial video to family with FMiner.

Attachments:
attachment p.fmpx (23.0 KB)

Offline

#5 April 23, 2014 13:13:30

matsrom
Registered: 2014-04-21
Posts: 8
Reputation: +  0  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

This is a great help! Thank you. But now my problem is that I want to capture content within the links on each page (enter the links of the companies and capture the e-mail, code, etc) and then return to de previous page to go to page 2.

To sum up, I don't know how to return to the previous page after having done click in a link of this one.
(I havn't found any tutorial about it).

Thank you for your help, this program seems to be very useful!

Attachments:
attachment Demo 2.fmpx (27.6 KB)

Offline

#6 April 24, 2014 04:58:10

webscraper
Registered: 2014-03-25
Posts: 32
Reputation: +  0  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

@matsrom

I suggest you to run first scrape to just collect the links. Once you have all the links you can use them in goto action and set scraping pattern to capture data by visiting each page.

See the attached demo project.

Attachments:
attachment demo.fmpx (21.4 KB)

Offline

#7 April 24, 2014 05:16:29

admin
Registered: 2012-03-15
Posts: 289
Reputation: +  1  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

Here's “open more link(s)” action for this situation, you can add this action as another child of a action. “open more link(s)” is a special action, actions can add more than one of it as children. It's some like “open link in new browser” in common browser. I've changed the project, see attachment.

Edited admin (April 24, 2014 05:24:10)

Attachments:
attachment p.fmpx (29.0 KB)

Offline

#8 Dec. 10, 2014 00:23:23

DminerJ
Registered: 2014-10-26
Posts: 2
Reputation: +  0  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

I have the same exact issue as @Matsrom had whereas I need to extract data within a loop of “next” pages, but the data is within the links itself. It appears from this thread that the only way to accomplish this is to actually first collect the links and then go back with all the links and extract the data using a separate project. While it appears this approach can work, I wonder if a more automated approach is possible. For example the only barrier that appears to prevent @Matsrom's code (and mine) from continuing to the next page and extracting the data is that there is no option to create a loop based on whether the first loop of extraction is complete. If there were, then the next step in the project would be to “click” the next link, wait 2000ms, then repeat the “Open links” recursively with group select again. Does such an option exist? Could this be a feature request?

Offline

#9 Dec. 15, 2014 20:49:42

Armen
Registered: 2014-12-15
Posts: 8
Reputation: +  0  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

I have same problem trying to go next page. bat i dont know how to do that can you help me please. Give me some example file to i can work on it. Here is the link http://www.getauto.com/for-sale/acura-1?zip=92620&fromyear=1995&toyear=2015&fromSRP=On i make a group links works fine bat only i can scrap 25 to 50 data from the page. I need to go page 2,3,4 Thanks

Offline

#10 Dec. 16, 2014 20:23:12

admin
Registered: 2012-03-15
Posts: 289
Reputation: +  1  -
Profile   Send e-mail  

Scrap pages without next button and return to the previous page

Armen
I have same problem trying to go next page. bat i dont know how to do that can you help me please. Give me some example file to i can work on it. Here is the link http://www.getauto.com/for-sale/acura-1?zip=92620&fromyear=1995&toyear=2015&fromSRP=On i make a group links works fine bat only i can scrap 25 to 50 data from the page. I need to go page 2,3,4 Thanks

This project is no different as this tutorial: http://www.fminer.com/scrape-multiple-yellow-pages-following-next-link/ Just add “openlinks” with “recursively” attribute can make it work. Try attachment.

Attachments:
attachment 333.fmpx (13.3 KB)

Offline

Board footer

Moderator control

Powered by DjangoBB