FMiner Web Scraping Video Tutorials
Here are web scraping tutorials for various types of sites and common skills of FMiner.
These tutorials are from simple to complex, and each tutorial has very few different steps as the previous one, so it's easy to be learned step by step.
Basic tutorials
Scrape Data from Page
These videos show how to scrape one or more sets of data on pages;
1. Scrape a set data from a page
The simplest video to show how to scrape a set of data from this page. [Project File]
2. Scrape multiple sets of data from a page
This video to show how to scrape multiple sets of data from this page. [Project File]
General Page with Links
These videos show how to scraping general sites(no Ajax), open links on pages; open recursive "next page" links.
1. Open links on a page then scrape data from these pages
Start from this page, then scrape data from the opened pages. [Project File]
2. Recursively open "Next" links, scrape data from these list pages
Start from this page, recursively open "Next" links, then scrape multiple set of data from these list pages. [Project File]
3. Recursively open "Next" links, open details links from these list pages, scrape data from the details pages
Start from this page, recursively open "Next" links, then open details links on these list pages, scrape data from the details pages. [Project File]
Dynamic(ajax) Pages with Buttons
These videos show how to scrape ajax sites without link, how to click buttons on pages, and click " next page" buttons recursively.
1. Click "next page" buttons(not links) recursively, scrape data from these list pages
Start from this page, click "next page" buttons recursively, scrape data from these list pages. [Project File]
2. Click "next page" buttons(not links) recursively, open details links on these list pages, scrape data from the details pages
Start from this page, click "next page" recursively, open details links on pages, scrape data from details pages. [Project File]
General Tips
Select Target(s)
Here's some skills to select targets, it's important to debug project and make project work stably(small changes of site will make the project still work).
1. Scrape table on pages
Scrape rows of the table on this page. [Project File]
2. Skills to Select Target Correctly
2.1 Change "Target XPath"
This video will show how to change target XPath.
2.2 Using "Relatively select" to Select Target Correctly
Scrape contents with "relatively select" on this page. [Project File]
Human actions for "login", "search"...
Video to show how to record human actions such as click, fill... on pages.
1. Fill data then click button
Fill data and click button on this page. [Project File]
Input Data
Videos to show how to input data on pages, it's important for seaching many keywords and scrape the results.
1. Input data 1 - Input data from data table
Input data from a table to this page. [Project File]
2. Input data 2 - Traverse and input every row of a data table
This tutorial based on previous video. [Project File]
Captcha
1. Resolve captcha on pages
Resolve captcha on this page. [Project File]
Regular Expression
1. Capture email address from pages with regular expression
Show how to scrape email from a page. [Project File]
Advanced Usage
1. Traverse all "option"s in a "select" control
This tutorial will show how to traverse "select" control on this page. [Project File]
2. Scrape full Ajax site with Nested Loop
This tutorial will show how to scrape full ajax site("next page" and "item" links are all buttons) with nested loop from this page. [Project File]