February 12, 2020

VirusTotal APK Malware Detection Data - Week 6: 20200203-20200209

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200203_20200209.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.83% 0.04% 32837 28 69113 57
Trustlook 99.65% 0.58% 32779 404 68737 115
ZoneAlarm 99.56% 0.01% 32749 4 69137 145
Kaspersky 99.31% 0.00% 32668 3 69138 226
DrWeb 98.02% 0.12% 32242 85 69056 652
Ikarus 97.96% 0.13% 32223 87 69054 671
Avira 97.64% 0.00% 32118 0 69141 776
F-Secure 97.60% 0.00% 32104 3 69138 790
AhnLab-V3 96.53% 0.04% 31752 27 69114 1142
Qihoo-360 95.47% 0.03% 31403 19 69122 1491
Sophos 94.39% 0.04% 31050 31 69110 1844
McAfee 92.90% 0.01% 30559 6 69135 2335
Tencent 84.12% 0.10% 27670 67 69074 5224
CAT-QuickHeal 83.98% 0.02% 27623 13 69128 5271
Avast-Mobile 78.36% 0.17% 25777 117 69024 7117
NANO-Antivirus 70.44% 0.02% 23169 15 69126 9725
AVG 64.48% 0.05% 21210 34 69107 11684
Cyren 62.65% 0.01% 20609 8 69133 12285
Avast 62.65% 0.05% 20608 34 69107 12286
McAfee-GW-Edition 57.98% 0.00% 19071 0 69141 13823
Fortinet 56.94% 0.00% 18729 0 69141 14165
Symantec 54.11% 0.03% 17799 21 69120 15095
K7GW 49.00% 0.07% 16119 48 69093 16775
MAX 44.97% 0.00% 14791 0 69141 18103
Rising 7.45% 0.03% 2449 21 69120 30445
TrendMicro-HouseCall 6.26% 0.01% 2058 10 69131 30836
Antiy-AVL 3.89% 0.01% 1280 4 69137 31614
BitDefender 2.61% 0.00% 859 0 69141 32035
Ad-Aware 0.09% 0.00% 30 0 69141 32864
Baidu 0.05% 0.01% 15 4 69137 32879
Babable 0.00% 0.00% 0 0 69141 32894
TotalGoodware 69141
TotalMalware 32894
TotalSample 102035

Please send an email to lxu@trustlook.com if you have any comments. Thanks.