Beta feature: New and more consistent crawler – 10 minute mail

Here at Disposable mail, we love new features, and we know that a lot of our customers do too. That’s why we want to give you a chance to try out what’s new as soon as we build it! Although they are new and exciting, beta features may not be 100% production-ready or may never even reach production for all customers.

Right now, you can enable beta features to try out our new crawler. The new crawler includes extended javascript rendering and even more actions triggered per page but we need help gathering more data and identifying potential bugs. If you’d like to give the new crawler a try, simply enable beta features by clicking on the slider in Advanced Settings.

Disposable mail beta features

Thank you for helping out! If you have any feedback or feature suggestions, don’t hesitate to get in touch at hello[at]detectify.com

Happy scanning!
//The Disposable mail Team

Temp Mails (https://tempemail.co/) is a new free temporary email addresses service. This service provide you random 10 minutes emails addresses. It is also known by names like: temporary mail, disposable mail, throwaway email, one time mail, anonymous email address… All emails received by Tempmail servers are displayed automatically in your online browser inbox.

New crawler enables smarter scans and more relevant findings – 10 minute mail

The crawler is the heart of the Disposable mail service. During the crawling phase, Disposable mail is trying to collect all possible pages of your website. That might seem like a simple task, and it used to be – back when the web was static. Now, more and more pages are generated dynamically, on the fly, by your customers as they visit your site.

Disposable mail has been working on an better and more thorough crawler for a long time and many of our awesome users tried out the new crawler when it was in beta and provided us with helpful feedback.

With this week’s release, we have now implemented the new crawler into our service, enabling you to run smarter scans and getting improved reports.

This is what the new crawler does:

  • To be able to reach better results, our crawler covers pages rendered by javascript.
  • The new crawler detect and collects dynamic pages much better than a regular crawler.
  • We have focused on consistency and more precise and in-depth scans, which can result in slightly longer scan times.  However, with better filtering, you most likely reduce the scan time.

Please note that due to improved crawling, your new findings might differ from old reports.

If you have any feedback or suggestions, go ahead and drop us a line at hello[at]detectify.com. We will continuously work on improving Detecitfy as a service, to make sure you can produce safer code.

Thank you for helping us improve our service!

Happy scanning!

Temp Mails (https://tempemail.co/) is a new free temporary email addresses service. This service provide you random 10 minutes emails addresses. It is also known by names like: temporary mail, disposable mail, throwaway email, one time mail, anonymous email address… All emails received by Tempmail servers are displayed automatically in your online browser inbox.

Hello Crawler 2.0! How we improved our core service and what this means for your scan results – 10 minute mail

If you follow our blog, you might have already seen an announcement introducing our updated core service with a new crawler. More reliable, more thorough, more deterministic, and with better coverage – sounds great, but what does it all mean? Find out how our engineers, who aim to build the world’s best and most thorough security scanner, have brought to life a new crawler that gives you even better results and helps you stay safe.

Why it was time to say goodbye to the old crawler

Natasha Lazarova - Disposable mail

Natasha Lazarova, software engineer

The crawler is where the fun begins; after gathering basic information, we crawl your website to identify the pages that will be tested. The quality of the entire scan depends on whether the crawler does its job well, so we wanted to improve crawling consistency and avoid duplicates. Because today’s web is not as static as it was in the past, our goals were to offer better javascript support and page filtering mechanisms.

The old crawler has served us well, but it was time to move on and build a crawler that is better-suited to our customers’ needs. Our software engineer Natasha Lazarova, who has been working on the crawler since April, says: “We started building the new crawler in spring, so it’s a result of more than half a year of hard work. It’s great to see that it’s performing well and that it has improved the quality of the entire service.”

 

Enter our shiny new crawler! What’s new?

Smart page filtering. This may very well be the biggest asset of the new crawler! How does it work? By looking at a few key metrics, for example

Crawler Visual - Disposable mail

Visual representation of our crawler in action

client state (cookies and other storage), dom structure and javascripts, we can filter properly. The smart page filtering has cut scan times, which is why your scan might take less time than it used to. Don’t worry, we are still carefully combing through your site, a shorter scan time simply means that smart page filtering is working its magic.

Improved javascript rendering support. We have drastically improved our javascript coverage, which means we can now render dynamically created DOM structures. Natasha explains: “Thanks to javascript rendering, we can now crawl corners of your website that we couldn’t crawl before.” Sorry Crawler 1.0, but this is something you simply couldn’t handle!

High level of configurability. The new crawler offers a bunch of new configuration options that we will be rolling out in the future for our power users. This includes options related to cookies, headers, connection timeouts, javascript timeouts, and allowing crawling actions (buttons/forms/links etc).

Improved crawl consistency. Consistent crawls are crucial for high quality scan results. “The crawling is now done sequentially, which makes it more consistent. Unless you have changed something on your website, two consecutive crawls should be the same,” Natasha says.

What’s next?

Now that the new crawler is up and running, Disposable mail’s engineers are working on optimizing other parts of the scanner: “The goal is to continuously work on making the scanner faster, produce better results and test for more vulnerabilities. We are also focusing on providing a higher level of transparency, such as the OWASP Top 10 view, where you can see how many tests you have passed or failed.”

Before we released the new crawler, it was available as a beta feature and tested by our awesome customers who kindly provided us with feedback. We would like to thank everyone who gave the new crawler a try while it was still in beta!

Temp Mails (https://tempemail.co/) is a new free temporary email addresses service. This service provide you random 10 minutes emails addresses. It is also known by names like: temporary mail, disposable mail, throwaway email, one time mail, anonymous email address… All emails received by Tempmail servers are displayed automatically in your online browser inbox.