September 23, 2019

VirusTotal APK Malware Detection Data - Week 38: 20190916-201909022

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20190916_20190922.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.62% 0.07% 41252 178 255223 159
Trustlook 99.51% 0.07% 41208 184 255217 203
K7GW 99.24% 0.08% 41098 215 255186 313
AhnLab-V3 98.52% 0.02% 40800 49 255352 611
Fortinet 98.34% 0.01% 40725 35 255366 686
Avast-Mobile 98.02% 0.11% 40590 282 255119 821
ZoneAlarm 97.52% 0.01% 40383 31 255370 1028
Avira 97.21% 0.00% 40255 3 255398 1156
Kaspersky 96.92% 0.00% 40135 6 255395 1276
Ikarus 96.05% 0.10% 39777 253 255148 1634
DrWeb 96.00% 0.11% 39755 290 255111 1656
F-Secure 94.35% 0.00% 39070 5 255396 2341
Qihoo-360 90.61% 0.01% 37523 29 255372 3888
Sophos 88.07% 0.01% 36470 36 255365 4941
McAfee 87.67% 0.00% 36306 5 255396 5105
Symantec 82.65% 0.03% 34227 72 255329 7184
NANO-Antivirus 82.27% 0.04% 34070 106 255295 7341
CAT-QuickHeal 82.09% 0.09% 33995 222 255179 7416
Tencent 76.07% 0.03% 31502 67 255334 9909
McAfee-GW-Edition 73.63% 0.00% 30489 3 255398 10922
AVG 73.33% 0.02% 30365 44 255357 11046
Avast 72.51% 0.02% 30026 42 255359 11385
Cyren 64.70% 0.00% 26791 6 255395 14620
MAX 59.87% 0.00% 24792 9 255392 16619
Rising 16.74% 0.01% 6933 16 255385 34478
TrendMicro-HouseCall 14.06% 0.39% 5822 996 254405 35589
Antiy-AVL 6.90% 0.01% 2857 21 255380 38554
BitDefender 3.86% 0.00% 1598 0 255401 39813
Baidu 0.39% 0.01% 161 27 255374 41250
Ad-Aware 0.36% 0.00% 151 0 255401 41260
Babable 0.00% 0.00% 0 0 255401 41411
Total Goodware 255401
Total Malware 41411
Total Sample 296812

Please send an email to lxu@trustlook.com if you have any comments. Thanks.