Blogs

How to crawl external links (on-site) in bulk?

If you're trying to capture all the external URLs like YouTube, links to other websites, or other calls to external sites that are happening from your front end website, then follow these steps to see all external URLs on-site in a spreadsheet.

  1. Open Screaming Frog.
  2. Type in the full URL to the website you want to crawl.
  3. Once the website is crawled 100% then in the Overview tab in the right-hand side pane, under "External" click "HTML" filter.
  4. Then in the results sort the "Address" field to organize the common websites together (to easily read).
  5. Highlight all items or just the websites you want to export to a spreadsheet (click the first one and then Shift-Click to highlight.
  6. Click Export
How to capture youTube video links on site.

You will now have a spreadsheet that shows the "From" page which is the page on your website the External URL is located. Then you will see "To" which is the External URL address that your page is linking/calling out to.

Handcrafted Migrations

Our service is a bit different than other automated systems. Our handcrafted migration solutions are designed to be handled with care, for well established companies, and to retain existing search engine rank (SEO).

All Migrations Include:

All Clients Receive Support 7-Days A Week
Ongoing Support
Project Management Portal
Scheduled Phone Calls
Quick Turnaround for Requests
Free Link QA Crawling
Strategic Site Launches
Contact Us
Copyright © 2021 | Pelican Commerce, LLC | All Rights Reserved
arrow-leftarrow-rightcheckmark-circle linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram