Last month, Google announced its support for the Global Signal Exchange, a new partnership between the DNS Research Federation (DNSRF) and the Global Anti Scam Alliance (GASA) focused on tackling online scams, fraud, and abuse. This partnership aims to reduce scams by 50% by 2030. 

While analyzing reports of abuse is fundamental to understanding the source of internet abuse, it only tells part of the story. It is also important to analyze and measure how, or even if, these reports of abuse are handled once discovered, in addition to understanding who mitigates them and how fast they are mitigated. 

In this article, we describe a new system developed at the DNSRF to track abuse mitigation rates and takedown times. For all abuse reports known to our DNS Analytics Platform, a.k.a. DAP.LIVE, we detect and then measure when a mitigation action occurs and how quickly it happens. In addition, we will determine and track which actor is responsible for each mitigation action. 

Ultimately, we will measure mitigation trends over time to understand the effectiveness and impact of not only the Global Signal Exchange partnership but also the new DNS abuse requirements on contracted parties as defined in the amendments to the Registrar Accreditation Agreement (RAA), and even the NIS2 regulation in the EU. 

Methodology Overview

What are we measuring

The inputs to this system are the abuse reports ingested and managed by the DAP.LIVE system from the following abuse block lists. 

• Phishing Reports: OpenPhish, APWG, Malware Patrol, URL Abuse, Scam Advisor
• Malware Reports: URLHaus, Malware Patrol, URL Abuse
  

On a typical day, we receive approximately 10,000 unique reports across all sources. The type of report varies by the input feed but is typically either a full URL, a hostname, or, in some cases, a domain name. Reports are deduplicated, and each report is enriched with relevant and timely data from multiple sources available on the DAP. 

Figure 1- Data Collection

The enriched data associated with each report include details of the website (HTTP status, website content, and metadata, etc.), domain name registration information (via RDAP/WHOIS), DNS information (name server and MX records), website security data (SSL/TLS certificate details), and data associated with the hosting provider (IP address, Asynchronous System (AS) Names and Numbers). 

Detecting and Categorizing Mitigation

The DAP detects possible mitigation by checking for what we call mitigation indicators. Each indicator hints at what kind of mitigation might have occurred when the mitigation occurred and the actor who might have made it. Note that some indicators are more indicative of actual mitigation than others, so the system distinguishes between what we believe to be high-confidence mitigations and those where mitigation is strongly suspected but hard to confirm. 

During this work's first phase, we will focus on the high-confidence mitigation indicators.

‎ ‎ ‎ ‎ ‎ ∙Registrar applied Client Hold (Registrar)

‎ ‎ ‎ ‎ ‎ ∙Domain entered RGP (Registrar)

‎ ‎ ‎ ‎ ‎ ∙Domain past expiry date (Registrar)

‎ ‎ ‎ ‎ ‎ ∙Registry applied Server Hold (Registry)
These indicators indicate with high confidence that mitigation has taken place and can be associated with at least one mitigation actor, allowing us to analyze which entities (who) are performing the mitigation. See the “What’s New” section below for details on how we plan to expand the set of mitigation indicators and actors we detect in future phases of this project. 

Figure 2 below visualizes the DAP mitigation detection and categorization mechanism. 

Figure 2 - Detecting and Categorizing Mitigation

1. Upon receipt of a new abuse report, perform an initial mitigation check. 
2. If a mitigation indicator has been detected, record it and stop future checks. 
3. If no mitigation has been detected, (*) check again on days 1, 3, 7, 14, 30, 90, and 365. If no mitigation is detected at 365 days, the record is marked as unmitigated, and no further checks are performed.

What are we counting, and why?

Mitigations are counted in two separate ways. The first counts the number of mitigated reports from URLs and host names. This gives us the overall mitigation picture of reports that can be actioned by multiple mitigators associated with different kinds of abuse. The second counts unique domain names with subdomain providers removed. This count allows for a more accurate and relevant analysis of mitigation rates and times for registrars and registries (independent and combined) within the scope of the new RAA obligations. 

Caveats of current implementation

As stated above, our findings are limited to abuse report data known to the DNSRF’s DAP.LIVE platform and based on the processing of approximately 10,000 phishing, malware, and scam reports received per day. We started collecting and processing this data in early September 2024. Our analysis is still young; thus, we expect our findings to evolve and improve over time. 

Phase 1 of our analysis will focus on mitigations that Registries and Registrars can action. Future phases will also track mitigations made by hosting providers, subdomain providers, and other actors. See the “What’s Next” Section for details.

Initial/Early Findings

The data in this section was live and accurate as of November 1, 2024. 

24-Hour Mitigation Rates

While we calculate mitigation rates for each of the mentioned intervals, in this article, we will focus on the 24-hour mitigation rate (e.g., the rate of mitigations within 24 hours of a report), which aligns with the review time required under the new RAA obligation.

Figure 3 - Mitigation rates over time for all reports

Figure 3 above shows the 24-hour mitigation rate trend from the beginning of September to the present. While the data's time frame is limited, it is worth noting that the mitigation rate for all reports of abuse is trending slightly upward, starting at around 3% and increasing to slightly under 10%. This is an encouraging sign. 

Figure 4 below shows the trends for mitigation rates of unique domain names where subdomain providers have been removed for the same period. 

Figure 4 - Mitigation rates over time for unique domain names (no subdomain providers)

Again, mitigation rates increase over time, which aligns with the previous visualization. 

Figure 5 shows the same period's 72-hour (3-day) mitigation rates for comparison.

Figure 5 - 3-Day Mitigation Rates

Figure 6 below shows the 7-day mitigate rates.

Figure 6 - 7-day Mitigation Rates

Mitigation Actors

Figure 7 shows that for the small percentage (less than 10%) of reports that appear to be mitigated, over 75% were mitigated by registrars, with registries mitigating approximately 20%. 

Figure 7 - Mitigation actors

For the remaining 5%, we found indicators applied by the registry and registrar. Given that registrars own the customer relationship with the registrant (and the abuser in these cases), the finding that registrars are responsible for most mitigations is unsurprising. 

This finding is similar no matter when we detected the mitigation. See Figure 8.

Figure 8 - Mitigation actors (x axis = days since report, y axis = number of reports)

Mitigation Times

Finally, when we look at the time between a report and a detected mitigation, we find that takedown times happen in a matter of days. (To calculate these figures, we only considered reports that had been checked for at least 14 days)

Figure 9 - Mitigation time comparison

While these times might seem reasonable, we know from previous research that most internet abuse, including Phishing and Malware attacks, lasts no more than 24 hours and sometimes only 2 hours before the attacker tears down the abusive infrastructure to cover their tracks. So, there is room for improvement to ensure verifiable instances of abuse are mitigated as quickly as possible to minimize the impact on internet users. It is also interesting to note that when registries are involved in mitigating abuse, their response time is quicker than that of registrars. 

What’s Next

Looking forward to the next phase of this work, we plan to ensure we see the full mitigation picture by analyzing the full range of mitigations beyond those done by registries and registrars. This includes mitigations done by hosting providers, subdomain providers, law enforcement agencies, registrants, browser vendors, etc. We view these indicators as lower confidence, as the changes below could be done for various reasons, some unrelated to abuse mitigation. Some of the mitigation indicators we are considering include

• Changes of Registrar (Registry/Registrar)
• Nameserver Changes (Registry/Registrar/Hosting Provider/Registrant)
• DNS Record Changes - A, AAAA Records Removed or Changed (Registrar/Hosting Provider/Registrant)
• Website status change - to not found or error statuses (Hosting Provider / Registrant)

Undoubtedly, these indicators will change and evolve as new mitigation methods appear in the ecosystem, and our systems will have to evolve along with them. We’ll be tracking any changes in response to the implementation of NIS2 in 2025 in light of the risk-based processes recommended by the NIS2 Cooperation Guidelines. In addition, we will determine the confidence level of each new mitigation indicator, as we believe some indicators are more difficult to confirm than others. We plan to have the system detect these additional mitigation indicators by Q2 2025. 

As we have done with other research projects, we will make all of these statistics and visualizations available to the public in the research section of our website by Q3 2025. This will include filtering and refining the stats based on abuse type, mitigation actor, and a combination of the two. Once we have a solid set of data available and analyzed, we also plan on publishing “league tables,” highlighting actors whose mitigation rates and times are the best and worst in the league. 

Conclusion

The DNSRF's new system for tracking abuse mitigation rates and takedown times provides a critical lens for evaluating the effectiveness of the Global Signal Exchange partnership and related industry and regulatory frameworks. By continuously monitoring mitigation actions across multiple actors and types of abuse, the system quantifies how abuse reports are being addressed and highlights areas where response times and accountability can be improved. Our initial findings underscore the role of registrars in handling the majority of mitigations while also revealing the need for more rapid intervention to keep up with abuse lifecycles that often end within 24 hours.

Key Takeaways

1. Significant Progress in Mitigation Tracking: DNSRF's tracking system offers real-time insights into abuse mitigation actions, adding a critical layer of visibility beyond abuse reporting alone.
2. Mitigation Rates are Low: Less than 90% of reports were mitigated within 24 hours of being reported, indicating the need for more work to achieve better rates. 
3. Room for Improvement in Response Times: While the average mitigation time spans a few days, faster responses are crucial, particularly for high-impact abuse like phishing and malware that often last less than a day.
4. Future Focus on Additional Mitigation Actors: To further enhance the ecosystem's ability to handle abuse, future phases of this work will expand tracking to include mitigations by hosting providers, law enforcement, registrants, and others, providing a fuller picture of the efforts against online scams and abuse.

These takeaways reinforce the importance of ongoing monitoring and adaptation as new abuse patterns and mitigation methods emerge, ensuring that all ecosystem players can respond effectively to protect internet users.