October 5, 2020

VirusTotal APK Malware Detection Data - Week 40: 20200928-20201004

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200928_20201004.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.86% 0.01% 8828 7 59540 12
DrWeb 98.60% 0.19% 8716 115 59432 124
Trustlook 98.31% 0.13% 8691 80 59467 149
Avira 98.29% 0.00% 8689 1 59546 151
K7GW 97.96% 0.12% 8660 72 59475 180
Fortinet 97.66% 0.02% 8633 10 59537 207
CAT-QuickHeal 96.87% 0.01% 8563 6 59541 277
Avast-Mobile 95.57% 0.21% 8448 123 59424 392
McAfee 95.03% 0.00% 8401 1 59546 439
ZoneAlarm 94.83% 0.00% 8383 1 59546 457
Kaspersky 94.16% 0.00% 8324 1 59546 516
Ikarus 93.90% 0.12% 8301 73 59474 539
AhnLab-V3 93.35% 0.04% 8252 24 59523 588
McAfee-GW-Edition 93.07% 0.01% 8227 4 59543 613
F-Secure 87.77% 0.00% 7759 0 59547 1081
NANO-Antivirus 75.59% 0.05% 6682 27 59520 2158
Sophos 74.93% 0.04% 6624 22 59525 2216
Symantec 69.21% 0.01% 6118 6 59541 2722
Qihoo-360 64.24% 0.03% 5679 15 59532 3161
AVG 42.27% 0.03% 3737 16 59531 5103
Ad-Aware 1.18% 0.00% 104 0 59547 8736
TotalGoodware 59547
TotalMalware 8840
TotalSample 68387

Please send an email to lxu@trustlook.com if you have any comments. Thanks.