Run a Google dork through the Custom Search API and export every result URL to a .txt file.
Query preview: —
Google rarely serves beyond ~300 results for a single query. For deeper coverage, narrow the dork (per-subfolder, per-keyword, or per-filetype) and merge the .txt files.