December 3, 2019

VirusTotal APK Malware Detection Data - Week 48: 20191125-201901201

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191125_20191201.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.92% 0.13% 19127 85 66098 16
K7GW 99.43% 0.74% 19033 493 65690 110
ZoneAlarm 99.20% 0.15% 18989 102 66081 154
Fortinet 99.13% 0.03% 18977 21 66162 166
Kaspersky 98.91% 0.15% 18934 102 66081 209
AhnLab-V3 98.68% 0.19% 18891 125 66058 252
DrWeb 98.35% 0.92% 18827 609 65574 316
Avira 98.12% 0.00% 18783 1 66182 360
F-Secure 97.90% 0.04% 18741 24 66159 402
Trustlook 96.91% 0.38% 18552 252 65931 591
Ikarus 96.74% 0.70% 18519 464 65719 624
McAfee 96.02% 0.02% 18381 11 66172 762
Symantec 85.48% 0.05% 16364 31 66152 2779
CAT-QuickHeal 82.13% 0.28% 15722 185 65998 3421
Avast-Mobile 78.52% 0.13% 15031 88 66095 4112
Qihoo-360 73.17% 0.10% 14007 66 66117 5136
Sophos 66.54% 0.04% 12738 28 66155 6405
NANO-Antivirus 59.33% 0.26% 11357 171 66012 7786
McAfee-GW-Edition 53.94% 0.00% 10326 1 66182 8817
AVG 31.85% 0.04% 6098 29 66154 13045
Avast 31.43% 0.04% 6016 28 66155 13127
Cyren 30.56% 0.01% 5850 6 66177 13293
MAX 25.51% 0.00% 4884 3 66180 14259
Tencent 18.99% 0.00% 3635 0 66183 15508
Rising 5.14% 0.03% 983 17 66166 18160
BitDefender 2.17% 0.00% 415 0 66183 18728
TrendMicro-HouseCall 1.98% 0.02% 379 14 66169 18764
Baidu 0.20% 0.01% 38 6 66177 19105
Antiy-AVL 0.09% 0.00% 18 0 66183 19125
Ad-Aware 0.04% 0.00% 8 0 66183 19135
Babable 0.00% 0.00% 0 0 66183 19143
TotalGoodware 66183
TotalMalware 19143
TotalSample 85326

Please send an email to lxu@trustlook.com if you have any comments. Thanks.