December 23, 2019

VirusTotal APK Malware Detection Data - Week 51: 20191216-201901222

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191216_20191222.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.52% 0.17% 22688 90 54283 109
Trustlook 99.48% 0.25% 22679 138 54235 118
ESET-NOD32 99.36% 0.03% 22651 19 54354 146
ZoneAlarm 98.96% 0.04% 22560 23 54350 237
Fortinet 98.77% 0.01% 22517 4 54369 280
Kaspersky 98.50% 0.04% 22456 23 54350 341
DrWeb 98.06% 0.12% 22354 66 54307 443
Ikarus 97.51% 0.17% 22229 90 54283 568
Avira 97.38% 0.00% 22200 1 54372 597
F-Secure 95.90% 0.00% 21862 2 54371 935
Sophos 93.55% 0.01% 21326 7 54366 1471
McAfee 92.92% 0.01% 21184 7 54366 1613
Qihoo-360 87.83% 0.05% 20023 28 54345 2774
Avast-Mobile 85.25% 0.18% 19434 98 54275 3363
Symantec 84.37% 0.04% 19233 22 54351 3564
AhnLab-V3 83.46% 0.07% 19027 37 54336 3770
CAT-QuickHeal 80.39% 0.06% 18326 32 54341 4471
NANO-Antivirus 76.12% 0.08% 17354 41 54332 5443
McAfee-GW-Edition 72.12% 0.00% 16441 2 54371 6356
AVG 70.50% 0.04% 16071 24 54349 6726
Avast 69.36% 0.04% 15811 24 54349 6986
Cyren 61.54% 0.00% 14030 2 54371 8767
Tencent 53.79% 0.00% 12263 0 54373 10534
MAX 51.36% 0.00% 11709 1 54372 11088
TrendMicro-HouseCall 13.48% 0.04% 3072 23 54350 19725
Rising 13.40% 0.02% 3055 11 54362 19742
BitDefender 7.04% 0.00% 1605 0 54373 21192
Antiy-AVL 6.14% 0.00% 1400 1 54372 21397
Ad-Aware 0.23% 0.00% 52 0 54373 22745
Baidu 0.13% 0.01% 29 4 54369 22768
Babable 0.00% 0.00% 0 0 54373 22797
TotalGoodware 54373
TotalMalware 22797
TotalSample 77170

Please send an email to lxu@trustlook.com if you have any comments. Thanks.