Screaming Frog and SEMrush are used by marketers to perform website audits that detect technical SEO issues. Fixing these issues improves website performance in Google's search results. It's critical to understand that Screaming Frog is a specialty tool used solely for website audits, while SEMrush is a general SEO solution.
SEMrush's website auditing tool is one of several tools that come with its monthly subscription.
Here's an overview of Screaming Frog and Semush in terms of features, ease of use, pricing, and who each solution is best for. Continue reading for an in-depth deep dive comparing both solutions. Table of Contents. Top Competitors. How Much Do Screaming Frog and SEMrush Cost?
As you can see in the Venn diagram below, both Screaming Frog and SEMrush come with basic crawling features. Screaming Frog includes many advanced crawling features that SEMrush does not offer. In addition to its web crawler, SEMrush comes with tools for keyword research, keyword rank tracking, competitor research, and more.
A complete breakdown of SEMrush's top features can be found in our SEMrush review. A web crawler is a software tool that analyzes the pages & structure of your website in order to identify technical SEO issues. Fixing these SEO issues will improve the performance of your website on Google and other search engines.
Common technical issues discovered by web crawlers include: . Pages with 300, 400 and 500 status codes . Missing HTML tags (title tag, canonical, H1s, meta description).
|App Name:||Screaming Frog|
|Developer:||Screaming Frog Ltd|
|Version:||Screaming Frog 11.3|
|Updated:||June, 22nd 2019|
|Category||Office and Business Tools|
|File Size:||247 MB|
|File Type:||EXE file|
|Requirement:||PC/laptop/tablet running Windows|
(Windows XP / Vista / Windows 7 / Windows 8 / Windows 10)
Duplicate content. Let's compare Screaming Frog to SEMrush's web rawler, the Site Audit tool. As you'll see, Screaming Frog includes advanced crawling & reporting features that larger companies & marketing experts require.
Meanwhile, SEMrush's Site Audit tool is more than sufficient for most SMBs, and is much easier to use.
- Screaming Frog and SEMrush have website crawlers that work very similarly.
- They test for the same core technical issues.
- In addition, Screaming Frog's spider has several advanced features, we'll share a few of our favorites below.
- Screaming Frog and SEMrush test for mostly the same website elements during their crawl.
Website Auditor vs Screaming Frog Review: General features
The major categories of website elements test for are: . Metadata: Are title tags, metadescriptions, H1s, H2s, alt tags and canonicals implemented properly? HTTP/HTTPS: Are pages and links HTTPS?
Directives: Are you properly using directives like Noindex and Nofollow? Status codes: Are pages returning proper 200 & 300 level status codes?
Are any returning 400 and 500 level errors? Duplicate content: Do you have duplicate pages, URLs, title tags or other content that is confusing search engines?
Page response time: Do your pages load quickly enough? Internal linking: What is the internal linking structure of your website?
- Image sizes: Are your image files too large, or the wrong image file format?
- Structured data: Have you properly implemented structured data markup to communicate that your pages are products, recipes, reviews or other standardized content types?
- Site structure: What is the hierarchy of your web pages?
- Do you have any high-value pages with poor page depth?
- Here's a few of the advanced features that Screaming Frog offers marketing experts.
- Finding orphaned pages: If you connect Google Analytics or Google Search Console to Screaming Frog, it will identify orphaned pages.
Crawling authenticated staging sites: Screaming Frog allows SEOs to crawl staging sites and development servers. This is useful for testing new websites still under development. Auditing redirects in a site migration: When a site is migrated to a new URL, it's critical that all 301 redirects are implemented properly.
Screaming Frog has created a report using its 'always follow redirects' feature that allows SEOs to see a detailed map of full redirect chains for any URL. Log file analysis is an advanced SEO tactic used to understand exactly how search engines like Google are crawling your website. Each time Google loads a page on your website, it creates a record. By aggregating these records, you can see which pages Google has crawled, in what order, and how often.
Log file analyses yield valuable information such as: . What pages Google crawls most. What valuable pages Google doesn't crawl.
- How much Google is crawling your website in total, and with each Googlebot.
- Both companies offer a log file analysis tool as an add-on: .
- Screaming Frog sells its Log File Analyser for £99.00 per year.
- SEMrush recently launched a Log File Analyzer that is free with its subscription. One of the biggest differences between Screaming Frog and SEMrush is reporting.
- Screaming Frog provides much more data, but has a steeper learning curve.
- Screaming Frog provides direct access to the crawl's raw data for every page.
- This is ideal for marketers who want to dive deep into crawl data and do their own analyses.
- The Screaming Frog application also provides access to far more data than SEMrush.
- In the H1 tab below, you can see H1s for all pages, and use filters to quickly find all pages with a missing or duplicate H1.
- SEMrush produces easy-to-interpret reports that are ideal for most SMBs.
- These reports remove guesswork by providing a summary of site health and a list of top errors that should be fixed.
Screaming Frog & SEMrush Feature Overview
- The SEMrush Compare Crawls report is a valuable feature that lets you quickly compare crawl results over time.
- This will help you understand if you are making progress fixing technical errors over time.
- SEMrush's Site Audit tool is easier to use than Screaming Frog.
- This makes sense – SEMrush's tool is built for SMBs that don't need advanced features or access to every data point.
- Screaming Frog has advanced features & robust reporting.
- Therefore, it has a steeper learning curve.
- For example, interpreting crawl results is easier in SEMrush. Its tool has a well-designed interface that highlights the key takeaways from the site audit.
- The overview tab includes an overall site health score, thematic reports, and a ranked list of the top issues.
- Compare this to Screaming Frog's Crawl Overview report, which is exported as a CSV file.
Website Auditor vs Screaming Frog Review: Technical site audit
- This report contains significantly more data than what you find in SEMrush, but you have to know what to look for and how to interpret it.
- SEMrush's interface also includes tool tips that you can scroll over to fully understand what you're looking at.
- Screaming Frog and SEMrush have different top competitors.
- Screaming Frog competes against other crawlers, while SEMrush competes most often against other all-in-one SEO solutions.
Screaming Frog competes with other web crawlers. Here's the best alternative web crawlers: .
- SEMrush competes most directly with other all-in-one SEO solutions.
- Here's the top SEMrush competitors: .
- For more detail, read our full guide to SEMrush competitors.
- SEMrush is significantly more expensive than Screaming Frog.
- However, the SEMrush subscription buys you around 30 SEO tools.
Screaming Frog is desktop software that is priced per license (1 user = 1 license). Free license to crawl up to 500 URLs. £149.00 per license per year. Bulk discounts for purchasing multiple licenses. No page crawl limit. SEMrush pricing starts at $119.95 per month.
All prices below are for one user: . Pro Plan: $119.95 per month. Guru Plan: $229.95 per month. Business Plan: $449.95 per month. Limits on number of URLs that can be crawled per month and per project. Screaming Frog and SEMrush are both excellent SEO solutions.
When comparing the two, remember that Screaming Frog is a specialty solution for website audits, and SEMrush is a general SEO solution with several use cases.
If you need to complete an in-depth site audit to identify & fix technical SEO issues, then Screaming Frog is an excellent option. We recommend evaluating Screaming Frog, Deepcrawl & Sitebulb. If you want an SEO tool that does a little of everything, and excels at keyword research, rank tracking & competitive analysis, then you'll love SEMrush.
We recommend comparing it against Ahrefs and SERPstat. Click here to download the free version of Screaming Frog. To start your free trial with SEMrush, click here. Screaming Frog is a very powerful SEO Spider able to perform in-depth SEO OnSite analysis.
In this guide, we will see some of the main features very useful during SEO analysis. The free version of Screaming Frog allows you to analyze up to 500 URLs.
Screaming Frog allows you to crawl a specific website, subdomain or directory. In the paid version, the SEO Spider allows you to select the “Crawl all Subdomain” option if you have more than one subdomain. If you only need to crawl one subdomain, simply add the URL in the appropriate box.
The most commonly used features are monitoring status queues on a website (40x,50x,200 and 30x). Screaming Frog by default crawls a directory by simply adding the address in the bar as presented in the image below. If you need to perform an advanced crawling you can use the wildcard that tells the SEO Spider to crawl all pages that precede and/or follow the “Wildcard”.
The path to use this feature is:. Spider > Include and add in the box that appears the desired syntax, for example with this syntax: https://www.bytekmarketing.com/about/.* the spider only crawls the sections of the website that are present in the “About Us” branch of the website, then all the resources that are after the “Jolly” character.
Starting the crawl will extract all the “Daughters” URLs of the “About Us” section, for example: https://www.bytekmarketing.com/about/roberto-paolucci or https://www.bytekmarketing.com/about/mario-rossi. This option is particularly useful with large websites where we do not have resources to work on very large data.
Keep in mind that the crawling data will have to be (in most cases) processed in Excel, so the starting point will have to be a workable data in an easy way to “Search Vert”, work with filters and charts.
From the “Mode” tab you can select the crawling mode, in case you want to crawl a set of URLs the mode to set is “List” because you can import an Excel file with a column containing the list of URLs. The other option to scan a list of URLs is “copy and paste”, then copy from an external source (Excel, CSV, TXT or HTML page) the list of URLs and click “Paste”.
It is necessary that each URL also contains the http or HTTPS protocol including the www, so the correct structure of each URL should be: http://www.test.it. When you need to analyze a large website and it’s not enough to just crawl through HTML and images (in SEO perspective very often it’s good to also analyze the status queues of CSS and JS files to make sure that search engine spiders are able to correctly render pages) you can work on the settings:.
1.Configuration > System > Memory and allocate more memory, for example 4GB2.Set the storage to the database instead of RAM.