Magento

The simplest way to detect bot traffic on Adobe Commerce.

The simplest way to detect bot traffic on Adobe Commerce

Last week Adobe Commerce team reported that a new feature became available - the Bots tab. It was created to allow the website owner to identify and monitor the bot traffic.


Nowadays, the Internet is browsed by many automated software pieces called bots. They may be helpful (such as search engine ones that add the website pages to the search results output) or malicious (spamming the sites, scraping content, etc.). So, knowing what bots are visiting your site is important and can help you avoid some website problems.


Due to the high importance, Adobe Commerce now allows us to see the bot traffic, which is a part of the Observation tool - a New Relic nerdlet. It shows the summary of non-cached bot activity, so it is easy to understand which load the bot causes on the site. Also, the user can see which bot activity causes site errors (quite often, it happens due to the load). In addition, the tab displays the bot names (recorded in User-Agent values) and IP addresses.


Having this information, the administrator can solve the issues caused by the bots in a few ways, such as using Fastly for rate-limiting, blocking the bot’s IP address or name values, or configuring robots.txt (applicable for law-abiding bots). In addition, Bing or Google search bots can be managed through the search engine console.


The list of the fields provided in the tab can help you understand how wide the range of the information provided by the tool is. It displays:

  • Total bot traffic by the name or IP address;
  • Bots with HTTP status errors (graph);
  • IPs that do not identify as bots with HTTP status errors (table);
  • Cache Status ‘ERROR’ detail table;
  • 5XX status distribution across IP addresses;
  • IP cache status (MISS, PASS, ERROR) and HTTP status;
  • Fastly Cache Summary;
  • IPs that do not identify as bots without error (graph);
  • Suspicious Non-Bot traffic (graph);
  • Blocked Bot name / IP addresses (in Fastly);
  • Blocked non-Bot name / IP addresses (in Fastly);
  • Number of user agents per IP address, number of successful, unsuccessful, and blocked requests (table);
  • IP addresses with errors, including and excluding 403;
  • Pageview Latency.

So, as you can see, the tool allows you to track much more than only bots, providing you with a better understanding of overall site performance, and allowing you to fine-tune it, so your customer can have a better experience. But if you think that’s too complicated, we can help you! With over 40 developers on our team, we can help you to resolve issues with the bots and much more, such as optimizing performance, integrating with third-party solutions, providing extension updates, quality checks, troubleshooting, bug fixes, security patches installation, and much more! Don’t hesitate to contact us!


In our blog, we post technology-related articles weekly. Follow us on Facebook and Instagram to get notifications about updates.

Reading next

Magento 2: 6 ways to increase sales
IT Service Providers: How To Choose The Right One in 2023?

Interested in a specific business & technology topic and looking for an article in our blog but haven't found one yet?

If you haven't come across an article that matches your query, feel free to suggest the topic to us, and we'll consider featuring it in our blog. Share your suggestion in the form below, and we'll be sure to review your request.

Talk with us

If you have any questions or problems in your business that can be solved with technical solutions, just let us know. We'll do everything we can to help you.