Highlighted features of FMiner
Here are some features of web scraping software - FMiner
Web macro recording for project build
You do not need to read or write any code, just record your actions of clicking and filling on the pages to build a project. All the actions are shown as diagrams in the scene, and can be changed according to your needs.
FMiner can record all kinds of actions that may be done on pages. Include:(e.g click, fill, check...), page actions(e.g page back, forward, refresh, clear cookies), so it can complete a variety of page operations to scrape any pages(include dynamic page with ajax).
Visual project designer
A project is shown as a flow chart in the designer, and every action is a diagram. You can drag and change them easily to complete a complex extraction task.
Scrape dynamic pages with no difference
FMiner do actions and scrape contents on pages just with recording and playing macro, and there is no different between static pages and dynamic pages(with ajax). FMiner will replay the actions correctly for all pages.
Capture any types contents
It can capture text, html, attribute(e.g href, class...) of a block on pages, pages url, metadata, download files, images on pages and extract contents with regular expression from pages' html code.
Export results to any formats
FMiner can export the scraped results to excel(.xls) and text(.csv) files. And it also can export the data to many database formats(e.g sql sever, mysql, postgres, access...) with ODBC engine, here you can assign your own connection string to make it support more database formats.
Scheduler and email report
FMiner support scheduler for tasks, and can report a mail after execution to show the results with the exported data as attachment.
Diagram links for flow control
Every action is a diagram in the designer, and they have two joint for flow control: when success do want, when fail do what. You can drag lines from the joints to link other diagram to assign the workflow to complete complex tasks.
In addition to these, FMiner has some special actions for advanced flow control: validate, run code.
Input data from table
FMiner can input data to controls on pages from a data table, so you can change the data in tables to scrape different pages without changing the project. For example, you build a project to scrape results from a page for searching a keyword from a data table, then you can change the table for other keywords.
Repeat input data from table
With the "read table" template, FMiner can input every row of a data table into pages for scraping all results of the data in table.
Adblock, image hide, plugin disable
FMiner has an adblock engine to block the unwanted contents, and you can change the filter in "adlist.ini" file.
FMiner can also hide images and disable plugins on pages to get good performance.
FMiner will open the links in multiple browsers automatically, when there are more than one links in "open link(s)" and "goto" actions.
Proxy and cookies manage for anonymous access
FMiner support change proxy or clear cookies before doing every action, so you can visit sites anonymously.
FMiner resolve captcha with a simple way, and can be done by manual inputting or some third party decaptcha service.
Embed python code
FMiner has a "run code" action can be added to project, and you can write python code in it to execute to control the workflow or operate the data tables.
Not only a web scraping software
FMiner is not only a web scraping software, it is also a visual web macro tool for recording and playing actions on pages. It is an alternative with full macro features for any other web macro software.