July 27, 2020

VirusTotal APK Malware Detection Data - Week 30: 20200720-20200726

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200720_20200726.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.80% 0.03% 28801 17 60417 59
K7GW 99.62% 0.08% 28751 51 60383 109
Fortinet 99.56% 0.03% 28734 20 60414 126
Trustlook 99.02% 0.23% 28578 140 60294 282
Avast-Mobile 98.71% 0.10% 28488 60 60374 372
Avira 98.68% 0.00% 28478 0 60434 382
DrWeb 98.40% 0.15% 28399 88 60346 461
AhnLab-V3 98.19% 0.03% 28338 17 60417 522
ZoneAlarm 96.97% 0.01% 27986 5 60429 874
Kaspersky 96.51% 0.00% 27852 0 60434 1008
F-Secure 96.24% 0.00% 27774 1 60433 1086
McAfee 96.17% 0.00% 27755 3 60431 1105
Ikarus 94.96% 0.10% 27405 61 60373 1455
Sophos 91.08% 0.02% 26287 14 60420 2573
Qihoo-360 87.76% 0.02% 25328 10 60424 3532
NANO-Antivirus 87.37% 0.04% 25215 24 60410 3645
Symantec 85.46% 0.01% 24664 4 60430 4196
AVG 79.77% 0.02% 23021 14 60420 5839
CAT-QuickHeal 56.57% 0.01% 16326 9 60425 12534
Ad-Aware 0.29% 0.00% 84 0 60434 28776
McAfee-GW-Edition 0.00% 0.00% 0 0 60434 28860
TotalGoodware 60434
TotalMalware 28860
TotalSample 89294

Please send an email to lxu@trustlook.com if you have any comments. Thanks.