Highlighted features of FMiner
Here are some features of web scraping software - FMiner
Web macro recording for project build
You do not need to read or write any code, just record your actions of clicking and filling on the pages to build a project. All the actions are shown as diagrams in the scene, and can be changed according to your needs.
FMiner can record all kinds of actions that may be done on webpages. These include: e.g click, fill, check..., page actions(e.g page back, forward, refresh, clear cookies), so it can complete a variety of page operations to scrape any pages(include dynamic page with ajax).
Visual project designer
A project is shown as a flow chart in the designer, and every action is visually shown the diagram. You can drag and change components of the diagram easily to edit or complete a complex extraction task.
Scrape dynamic pages with no difference
FMiner do actions and scrape contents on pages just with recording and playing macro, and there is no different between static pages and dynamic pages(with ajax). FMiner will replay the actions correctly for all pages.
Capture any type of content
It can capture text, html, attribute(e.g href, class...) of a block on webpages, pages url, metadata, download files automatically, images on pages and extract contents with regular expression from pages' html source code.
Export results to a variety of popular formats
FMiner can export the scraped results to excel(.xls) and text(.csv) files. And it also can export the data to many database formats(e.g sql sever, mysql, postgres, access...) with ODBC engine, here you can assign your own connection string to make it support more database formats.
Scheduler and email report
FMiner support scheduler for tasks, and can send a report mail after execution to show the results with the exported data as attachment for confirmation.
Diagram links for flow control
Every action is shown as a diagram in the designer, and they have two joint for flow control: when success perform: do what; when fail: do what. You can drag lines from the joints to link other diagram to assign the workflow to complete complex tasks.
In addition to these amazing features, FMiner has some special actions for advanced flow control: validate, run code.
Input data from table
FMiner can input data to controls on pages from a data table, so you can change the data in tables to scrape different pages without changing the project. For example, you build a project to scrape results from a page for searching a keyword from a data table, then you can change the table for other keywords. This has many practical, real world functions for a variety of projects.
Adblock, image hide, plugin disable
FMiner has an adblock engine to block the unwanted contents, and you can change the filter in "adlist.ini" file.
FMiner can also hide images and disable plugins on pages to increase performance levels of your project.
Repeat input data from table
With the "read table" template, FMiner can input every row of a data table into pages for scraping all results of the data in table.
FMiner will open the links in multiple browsers automatically, when there are more than one links in "open link(s)" and "goto" actions.
Proxy and cookies manage for anonymous access
FMiner supports change proxy or clear cookies before doing every action, so you can visit sites anonymously.
FMiner resolve captcha with a simple way, and can be done by manual inputting or some third party decaptcha service.
Embed python code
FMiner has a "run code" action can be added to project, and you can write python code in it to execute to control the workflow or operate the data tables.
Not just a web scraping package
FMiner is not only a web scraping software, it is also a visual web macro tool for recording and playing actions on pages. It is an alternative with full macro features for any other web macro software.