November 18, 2019

VirusTotal APK Malware Detection Data - Week 46: 20191111-201901117

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191111_20191117.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.70% 0.10% 29747 261 273699 89
Trustlook 99.19% 0.47% 29594 1280 272680 242
K7GW 98.92% 0.23% 29515 620 273340 321
AhnLab-V3 98.07% 0.03% 29261 92 273868 575
Avira 97.70% 0.00% 29151 4 273956 685
McAfee 97.51% 0.01% 29094 19 273941 742
Avast-Mobile 95.78% 0.11% 28578 288 273672 1258
Ikarus 94.88% 0.28% 28309 780 273180 1527
ZoneAlarm 92.50% 0.04% 27597 104 273856 2239
F-Secure 92.38% 0.00% 27563 9 273951 2273
DrWeb 92.03% 0.59% 27459 1606 272354 2377
Kaspersky 88.66% 0.01% 26454 35 273925 3382
Symantec 88.19% 0.06% 26311 175 273785 3525
Fortinet 83.73% 0.02% 24981 50 273910 4855
Qihoo-360 83.04% 0.03% 24776 69 273891 5060
NANO-Antivirus 69.86% 0.11% 20842 315 273645 8994
McAfee-GW-Edition 68.35% 0.00% 20393 1 273959 9443
Sophos 67.64% 0.07% 20182 184 273776 9654
CAT-QuickHeal 57.37% 0.23% 17117 639 273321 12719
MAX 49.14% 0.00% 14661 10 273950 15175
AVG 38.46% 0.04% 11475 97 273863 18361
Avast 36.41% 0.03% 10863 89 273871 18973
Cyren 31.56% 0.01% 9417 27 273933 20419
Tencent 21.11% 0.00% 6299 0 273960 23537
Antiy-AVL 4.11% 0.01% 1225 26 273934 28611
Rising 3.53% 0.02% 1054 49 273911 28782
BitDefender 2.53% 0.00% 755 0 273960 29081
TrendMicro-HouseCall 2.36% 0.04% 704 122 273838 29132
Ad-Aware 0.08% 0.00% 25 0 273960 29811
Baidu 0.07% 0.01% 22 33 273927 29814
Babable 0.00% 0.00% 0 0 273960 29836
TotalGoodware 273960
TotalMalware 29836
TotalSample 303796

Please send an email to lxu@trustlook.com if you have any comments. Thanks.