• You are not logged in.

#1 Jan. 7, 2015 14:25:37

Alecwhankins
Registered: 2015-01-07
Posts: 5
Reputation: +  0  -
Profile   Send e-mail  

Open links, move to next page, open links, etc.

Hello,
I am having trouble moving on to the “next page” to continue opening/scraping links.

I am trying to scrape contact information for lawyers within this directory. e.g. opening their profile links and moving page-to-page.

I have successfully opened/scraped the links on the first page, but haven't been able to continue on to the next ones.



The URL I am scraping is: https://services.utahbar.org/Member-Directory
Search: “Utah” as the state.


Per the tutorial, I set up my scrape flow like this:

PLEASE HELP!

Offline

#2 Jan. 13, 2015 08:02:54

admin
Registered: 2012-03-15
Posts: 289
Reputation: +  1  -
Profile   Send e-mail  

Open links, move to next page, open links, etc.

I found the “next” link is in iframe of the page, so the new opened pages are not same as the original page. Here's 2 ways to deal with this problem:
1. You can make the project starting from https://services.utahbar.org/cvweb/cgi-bin/utilities.dll/CustomList?STATECD=UT&SORT=LASTNAME,%20FIRSTNAME&RANGE=1/25&SQLNAME=UTMEMDIR&WHP=Customer_Header.htm&WBP=Customer_List.htm

2. Or make a project with loop clicking like this tutorial: http://www.fminer.com/23-click-next-page-button-ajax-pages/

I've made a project of the second way in attachment. But I still comment follow the first way.

Attachments:
attachment utah.fmpx (23.1 KB)

Offline

Board footer

Moderator control

Powered by DjangoBB