September 16, 2019

VirusTotal APK Malware Detection Data - Week 37: 20190909-201909015

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20190909_20190915.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.37% 0.08% 66395 494 581132 420
K7GW 99.32% 0.18% 66361 1044 580582 454
Trustlook 98.85% 0.11% 66047 661 580965 768
Avira 98.55% 0.00% 65848 8 581618 967
Avast-Mobile 96.32% 0.16% 64355 942 580684 2460
AhnLab-V3 95.54% 0.02% 63832 130 581496 2983
McAfee 95.48% 0.00% 63798 4 581622 3017
ZoneAlarm 93.53% 0.03% 62491 149 581477 4324
Fortinet 92.99% 0.01% 62131 69 581557 4684
Ikarus 91.38% 0.19% 61053 1101 580525 5762
Symantec 91.24% 0.02% 60961 136 581490 5854
F-Secure 90.94% 0.00% 60761 21 581605 6054
Kaspersky 89.58% 0.01% 59851 40 581586 6964
DrWeb 89.41% 0.36% 59740 2065 579561 7075
Qihoo-360 88.30% 0.01% 58998 74 581552 7817
NANO-Antivirus 80.78% 0.07% 53974 431 581195 12841
Sophos 77.73% 0.03% 51935 193 581433 14880
McAfee-GW-Edition 76.27% 0.00% 50962 11 581615 15853
Tencent 75.47% 0.02% 50422 140 581486 16393
CAT-QuickHeal 70.36% 0.18% 47009 1037 580589 19806
AVG 47.22% 0.03% 31551 149 581477 35264
Avast 46.02% 0.02% 30749 143 581483 36066
MAX 45.17% 0.00% 30183 17 581609 36632
Cyren 43.86% 0.01% 29304 63 581563 37511
Rising 5.36% 0.01% 3581 58 581568 63234
Antiy-AVL 4.57% 0.01% 3052 75 581551 63763
TrendMicro-HouseCall 3.33% 0.02% 2223 116 581510 64592
BitDefender 2.74% 0.00% 1830 0 581626 64985
Baidu 0.09% 0.03% 61 168 581458 66754
Ad-Aware 0.08% 0.00% 53 0 581626 66762
Babable 0.00% 0.00% 2 2 581624 66813
Total Goodware 581626
Total Malware 66815
Total Sample 648441

Please send an email to lxu@trustlook.com if you have any comments. Thanks.