This is a feature request board for Ultimate Web Scraper. Feel free to drop your suggestions here 📥 For support, please contact us via email support @ ultimatewebscraper.com

Arthur 2 months ago

Seems sites built on Webflow don’t provide a page URL for listing pages (cards) when running a List Extraction. They seem to be using a CSS class, but I can’t identify the right class to add to a List Extraction manually, resulting in no URLs collected. As a result, Page Extractions fail. Here’s an example of a site causing problems: hotel list

MrKhaki 5 days ago

I’ve finally identified the problem with large page extractions and failures… The browser tab memory grows dramatically during the scrape and drops extractions to a crawl with multiple page timeout errors. Recently on a long page extraction of 600 pages, after about 75 pages, Chrome starts slowing down (page reloads), then page timeouts. A look at Chrome’s Task Manager shows that for the tab being used memory usage gets out of control. In this case, over 8GB and growing. THE SOLUTION (but requires monitoring): When you first notice Chrome (or Edge) starting to reload a page, open a new tab. Then as soon as the content highlights for extraction, close it. This extracts the last content, then has UWS load the next page (and subsequent pages) in the open tab. This flushes the tab memory and page extractions resume like you’ve just started. Unfortunately, this might need to be done multiple times for a large list. If there was a way to clear the tab memory during the page extraction to get rid of the “previous/next” tabs accumulating during a page extraction, it would fix all the problems with memory/sluggishness and fail page extractions. Still hoping for failed page extractions to be re-run at the end of an extraction. Seems an error list could be added at the end of an extraction to accomplish this.

MrKhaki 5 days ago

Would love to see the ability to create projects within the browser app. This would enable the grouping of lists/pages (recipes could be part of this), but more importantly Pages. I tend to just run list and page extractions in succession and then later save the data externally. This approach requires me to keep a list of extractions that I then need to save. A project would keep website/topic extractions separate and easily scan in Data and export. Love this application and your openness to suggestions.

MrKhaki 5 days ago

Hi, I’d like to suggest an improvement for pagination scraping. Currently, when Ultimate Web Scraper clicks “View more” / “Load more” and new hotel results are loaded, it appears to scan the entire result list again from the beginning. This causes previously extracted records to be processed repeatedly and makes large result sets slower and less efficient. It would be very useful if the scraper could keep track of already extracted items and, after pagination or “Load more”, process only the newly loaded records. For example: First load: extract hotels 1–20 Click “Load more” Second load: extract only hotels 21–40 Click “Load more” Third load: extract only hotels 41–60 Instead of re-scanning hotels 1–20, then 1–40, then 1–60 each time. A deduplication or “process only newly loaded elements” option for pagination/infinite scroll would make large scraping jobs significantly faster and reduce unnecessary page processing.

Ozge Arslan 9 days ago

can add a grocery type incase you need product barcodes and a Email Receipt Extractor? That would be awesome if you made it!

Diodagoat 28 days ago

Can you extract laptop configurations and prices for multiple laptop models and configs from the apple website?

RK K about 1 month ago

the shopify detection banner setting always reverts to on after turning off in settings. Probably doesn’t affect anything, but since it’s an option, there must be a reason to offer.

MrKhaki about 1 month ago

1

Is it possible to add download images to zip file, now it ask me to download and approved it one by one. in addition, can it be added to the excel/csv in the first column?

danielshaki about 1 month ago

1

If I pause a page extraction and later go into Data, there is a “Live” spinning status button (which can be paused). What is actually happening? Is it still extracting? My fans spinup in this view, but no new data is being added as far as I can tell.

MrKhaki about 1 month ago

It would be useful to have the option to pause and resume a page extraction in the browser extension, especially for large lists. I would think this is possible given the sequential order of extractions from the list of URLs. This would allow the free-ing up of the browser for other uses, but also could help see more successful scrapes. Still would like to see an option to re-run extractions for page timeouts. This, coupled with pause/resume would help see 100% data extraction.

MrKhaki about 2 months ago

With the new data list, please re-add “List” and “Pages” notation to the dropdown list. It takes a lot more attention to find the right data set without that information available. In dark mode, the dropdown list appears in light mode. The text in the dropdown could be a little larger. The new settings features are nice, but can you add a “default” setting that mimics the display before the new features?

MrKhaki 2 months ago

1