FMiner Main Interface


Main interface of FMiner consists of four parts.


The Flowchart Process Map Panel


With each step, you increment the flowchart and can follow the logical flow of your project. Each node in the scene is an action, and the starting action should be "goto" to open a URL.  The left actions formed the execution process, and the program will execute each of them following the arrow,  and the blue "scrape page" node containing some "capture content" nodes is for scraping data from the page.  


The Attribute Panel


This panel will show all attributes of the selected action. To change an action, you can select the node in the scene, then change its attributes here, such as Location(XPath) of the targets. Different types of actions have different attributes, they will show here dynamically.


The Integrated Browser Panel


Having the browser integrated into the main screen admin screen of the software is critical to a cohesive design cycle. When begin "record", you can do your actions here to build flowchart, and you can assign the location of targets(links, contents to capture...) by clicking DOM in it.


The Input and Output Panel


Having used the Browser, Process Map and Attribute Panels to define your project, you can view the resulting code, output results and input values tables in the Input Output Panel.

Note that you have four tabs at the bottom of this panel: Logs, Data, Selection, Variables.


Log Tab 


The Log Tab gives you an under the hood view of the project as the program steps through your web scraping instruction set to extract data elements from your target website.


Data Tables Tab 


This tab is used to access the table view of the extracted values and the form entry tables where other input variables such as target site URL lists, keywords for searching. Tables may be created and attributes defined, edited or deleted.


Selections Tab


Reflecting selections made throughout the browser and the attributes panel, this tab area displays the DOM codes you selected on the browser, thereby granting you a very precise lens into the DOM elements examined and selected at any step. Here's 3 subtables:

  1. Results: Just valid for "openlink(s)", "capture content" actions. To show the final URLs, caputured data.
  2. Selected blocks: The selected DOMs.
  3. Block code: The HTML code of selected DOMs.


Variables Tab


This tab will show all global variables in FMiner. Variable is for advanced usage to control execution process, such as a counter for a loop. You can make/change a variable in "run code" with code like this:

variables['counter1'] = 0