VirusTotal APK Malware Detection Data - Week 49: 20201130-20201206

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201130_20201206.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.49% 0.01% 19870 8 67635 102
Avast-Mobile 99.33% 0.25% 19838 172 67471 134
Avira 99.20% 0.00% 19813 1 67642 159
Trustlook 99.13% 0.16% 19799 106 67537 173
DrWeb 98.75% 0.21% 19723 141 67502 249
K7GW 98.72% 0.14% 19716 92 67551 256
Fortinet 98.31% 0.01% 19635 8 67635 337
ZoneAlarm 97.99% 0.00% 19570 1 67642 402
CAT-QuickHeal 97.77% 0.02% 19527 12 67631 445
Kaspersky 97.50% 0.00% 19473 0 67643 499
AhnLab-V3 96.30% 0.02% 19234 16 67627 738
Ikarus 95.92% 0.13% 19157 91 67552 815
F-Secure 92.11% 0.00% 18396 2 67641 1576
McAfee 84.31% 0.00% 16838 2 67641 3134
Sophos 82.94% 0.01% 16565 9 67634 3407
McAfee-GW-Edition 79.25% 0.01% 15828 8 67635 4144
NANO-Antivirus 79.11% 0.03% 15799 18 67625 4173
Qihoo-360 75.09% 0.01% 14997 5 67638 4975
AVG 70.23% 0.04% 14026 24 67619 5946
Symantec 65.94% 0.01% 13170 9 67634 6802
Ad-Aware 0.71% 0.00% 141 0 67643 19831
TotalGoodware 67643
TotalMalware 19972
TotalSample 87615

Please send an email to lxu@trustlook.com if you have any comments. Thanks.