October 28, 2019

VirusTotal APK Malware Detection Data - Week 43: 20191021-201901027

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191021_20191027.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
Trustlook 99.72% 0.33% 52153 545 163570 147
ESET-NOD32 99.71% 0.03% 52147 57 164058 153
K7GW 99.48% 0.06% 52030 103 164012 270
Avira 98.65% 0.00% 51595 0 164115 705
ZoneAlarm 97.68% 0.02% 51088 28 164087 1212
Kaspersky 97.08% 0.00% 50773 7 164108 1527
Avast-Mobile 96.64% 0.10% 50544 161 163954 1756
DrWeb 96.02% 0.10% 50216 166 163949 2084
Ikarus 95.95% 0.11% 50181 182 163933 2119
F-Secure 95.61% 0.00% 50006 2 164113 2294
McAfee 94.89% 0.01% 49625 9 164106 2675
AhnLab-V3 94.64% 0.02% 49497 36 164079 2803
Fortinet 91.03% 0.01% 47607 15 164100 4693
Symantec 84.30% 0.03% 44091 53 164062 8209
NANO-Antivirus 81.67% 0.02% 42715 38 164077 9585
Qihoo-360 80.58% 0.02% 42142 36 164079 10158
Sophos 77.44% 0.01% 40499 19 164096 11801
McAfee-GW-Edition 74.37% 0.00% 38898 2 164113 13402
CAT-QuickHeal 69.94% 0.05% 36578 85 164030 15722
AVG 49.27% 0.03% 25768 56 164059 26532
Avast 48.07% 0.03% 25139 54 164061 27161
Cyren 44.01% 0.00% 23017 4 164111 29283
MAX 42.94% 0.00% 22457 7 164108 29843
Tencent 30.54% 0.00% 15972 0 164115 36328
Rising 6.01% 0.00% 3144 2 164113 49156
Antiy-AVL 4.40% 0.01% 2301 13 164102 49999
TrendMicro-HouseCall 4.05% 0.15% 2116 247 163868 50184
BitDefender 2.72% 0.00% 1423 0 164115 50877
Ad-Aware 0.16% 0.00% 86 0 164115 52214
Baidu 0.07% 0.01% 37 21 164094 52263
Babable 0.00% 0.00% 2 2 164113 52298
TotalGoodware 164115
TotalMalware 52300
TotalSample 216415

Please send an email to lxu@trustlook.com if you have any comments. Thanks.