Spider

This feature is limited to 50 pages for free users. Pro Users enjoy unlimited page crawling.


Purpose

With the Spider you can combine functions in SeoTools into a powerful webpage crawler.


Features

  • Number of Hits - Decide how many pages you want to crawl.

  • Search Depth - Decide how "deep" you want o crawl. "2" means two layers of Sub Pages.

  • External Links - Decide if you want to include pages from external domains

  • Mode - Decide if you want to crawl web pages (Spider) or local files (List)


Functions

  • Email(URL)
  • Google AdSense Id(URL)
  • GoogleAnalyticsId(URL)
  • GoogleCacheDate
  • GooglePageSpeed(URL).Score
  • GooglePageSpeed.GoogleMobileReady(URL)
  • HtmlCanonical
  • HtmlH1
  • HtmlH2
  • HtmlH3
  • HtmlMeta
  • HtmlMetaDescription
  • HtmlMetaKeywords
  • HtmlMetaRefresh
  • HtmlTitle
  • HttpHeader
  • HttpStatus
  • IsFoundOnPage
  • LinkCount
  • Majestic.CitationFlow
  • Moz.Metrics(URL(s)).Canonical URL
  • PageCodeSize
  • PageCodeToTextRatio
  • PageEncoding
  • PageSize
  • PageTextSize
  • RegexpFindOnUrl
  • ResponseTime
  • SocialUrlScrapers.SocialMetaTags(URL).OGTitle
  • SocialUrlScrapers.TwitterAccount(URL)
  • SocialUrlShares.GooglePlusOnes(URL)
  • SocialUrlShares.LinkedInShares(URL)
  • SocialUrlShares.PinterestPinned(URL)
  • SocialUrlShares.StumbleUponLikes(URL)
  • SocialUrlShares.StumbleUponStumbles(URL)
  • Url
  • Url.AnchorText
  • Url.DiscoveredOn
  • Url.Error
  • Url.Internal/External
  • Url.Level
  • Url.Rel
  • Url.Type
  • UrlProperty.Absolute
  • W3CValidate
  • WordCount
  • XPathOnUrl

See

Guides

Related Functions


Get help with this function in the community →