Share your repls and programming experiences

← Back to all posts
Python Web Searcher
Geocube101 (590)

Some thing I made to search a page for keywords. Any links in the page are stored and searched as well. It's highly recommended to set the maximum amount of pages to search.

When running the program, you will be prompted for different things:
URL: The website URL
Use Page Authentication: Whether or not to use authentication (y or n)
-- Username: Username field for target site
-- Password: Password field for target site
Use Continuous Searching: Whether or not the scraper page count is capped
Flatten Output: Whether or not to flatten the output
Enter Target: Purpose of the scraper
-- element - Targets an html element tag to scan
-- string - Targets a string to scan
-- reachtest - Continuously scans until end of links list

Geocube101 (590)

Thanks @PDanielY
I'm surprised you found this; it's a year old

PDanielY (1074)

@Geocube101 oh I was looking through random posts and found this