How to judge it got last page when loop clicking "next" button or recursively opening "next" link


When making a project with "next" button or link, we should be careful for the last page. Generally, when it got to the last page, the "next page" button/link will disapear, the flow will stop.

But for some special pages, the "next page" button/link not disapear, may just change color to gray, or have no changes. For these kind sites.

1. The button changed on the last page

You should go to the last page, check the button's html code, to see the different of the button code on normal pages and the last page.  For example, you may see here's a "disabled" in the code of button on the last page. So you can change XPath, and add a condition not(@disabled) the XPath of button may this:

//button[contains(@class,'forward-btn') and not(@disabled)]

This means, when the button has "disabled" attribute, FMiner will can't find the target. Other example: here's a 'display: none' in style of the button DOM on the last page,  so add condition:

not(contains(@style,'display: none'))


2. The button no change on the last page

If here's no different of the button code on the last page and normal pages, it's some difficult to judge it got to the last page. Here's two way to resolve the issue:

1. First, find total page count. If the "next page" is button, add a counter with the total pages count in the loop. If the "next page" is link, and you use "Open link(s) recursively" for changing page, set "Max recursive level" to the total pages count.

And you should change the counter/ "Max recursive level"  manually when the page count changed.


2. This way is just for the "next page" is button.  In the loop, add a “scrape page” node to scrap “current page number” and “total pages count” to a temporary table. Add a run code action, and write some code to judge whether the current page number equal the total page count, if they are equal, break the loop, the run code may like this:

table = tables['temp']
row = table[len(table)-1]
if row['curPage'] == row['totPage']:
    result = False