FMiner Main Interface

 

Main interface of FMiner consists of four parts.

 

The Flowchart Process Map Panel

 

With each step, you increment the flowchart and can follow the logical flow of your project. Each node in the scene is an action, and the starting action should be "goto" to open a URL.  The left actions formed the execution process, and the program will execute each of them following the arrow,  and the blue "scrape page" node containing some "capture content" nodes is for scraping data from the page.  

 

The Attribute Panel

 

This panel will show all attributes of the selected action. To change an action, you can select the node in the scene, then change its attributes here, such as Location(XPath) of the targets. Different types of actions have different attributes, they will show here dynamically.

 

The Integrated Browser Panel

 

Having the browser integrated into the main screen admin screen of the software is critical to a cohesive design cycle. When begin "record", you can do your actions here to build flowchart, and you can assign the location of targets(links, contents to capture...) by clicking DOM in it.

 

The Input and Output Panel

 

Having used the Browser, Process Map and Attribute Panels to define your project, you can view the resulting code, output results and input values tables in the Input Output Panel.

Note that you have four tabs at the bottom of this panel: Logs, Data, Selection, Variables.

 

Log Tab 

 

The Log Tab gives you an under the hood view of the project as the program steps through your web scraping instruction set to extract data elements from your target website.

 

Data Tables Tab 

 

This tab is used to access the table view of the extracted values and the form entry tables where other input variables such as target site URL lists, keywords for searching. Tables may be created and attributes defined, edited or deleted.

 

Selections Tab

 

Reflecting selections made throughout the browser and the attributes panel, this tab area displays the DOM codes you selected on the browser, thereby granting you a very precise lens into the DOM elements examined and selected at any step. Here's 3 subtables:

  1. Results: Just valid for "openlink(s)", "capture content" actions. To show the final URLs, caputured data.
  2. Selected blocks: The selected DOMs.
  3. Block code: The HTML code of selected DOMs.

 

Variables Tab

 

This tab will show all global variables in FMiner. Variable is for advanced usage to control execution process, such as a counter for a loop. You can make/change a variable in "run code" with code like this:

variables['counter1'] = 0