October 21, 2019

VirusTotal APK Malware Detection Data - Week 42: 20191014-201901020

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191014_20191020.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.51% 0.06% 34573 74 130346 170
Trustlook 98.50% 0.14% 34223 183 130237 520
Avast-Mobile 98.41% 0.11% 34191 147 130273 552
Avira 98.23% 0.00% 34127 1 130419 616
AhnLab-V3 97.63% 0.02% 33920 30 130390 823
ZoneAlarm 96.23% 0.03% 33432 41 130379 1311
McAfee 95.76% 0.01% 33269 12 130408 1474
K7GW 95.00% 0.11% 33006 144 130276 1737
F-Secure 94.91% 0.00% 32974 3 130417 1769
DrWeb 94.90% 0.20% 32971 266 130154 1772
Ikarus 94.79% 0.14% 32934 182 130238 1809
Kaspersky 94.42% 0.02% 32804 22 130398 1939
Fortinet 93.07% 0.02% 32337 27 130393 2406
Symantec 87.44% 0.05% 30379 69 130351 4364
NANO-Antivirus 86.80% 0.03% 30156 45 130375 4587
Qihoo-360 82.76% 0.02% 28753 31 130389 5990
McAfee-GW-Edition 73.44% 0.00% 25514 3 130417 9229
Sophos 71.32% 0.02% 24780 27 130393 9963
CAT-QuickHeal 65.09% 0.09% 22613 114 130306 12130
Tencent 57.73% 0.05% 20056 63 130357 14687
MAX 44.07% 0.01% 15310 7 130413 19433
AVG 43.04% 0.04% 14952 46 130374 19791
Avast 41.74% 0.03% 14502 43 130377 20241
Cyren 39.67% 0.00% 13781 3 130417 20962
Antiy-AVL 5.33% 0.01% 1853 9 130411 32890
Rising 4.10% 0.01% 1424 8 130412 33319
TrendMicro-HouseCall 3.17% 0.11% 1103 146 130274 33640
BitDefender 3.02% 0.00% 1048 0 130420 33695
Baidu 0.09% 0.02% 33 28 130392 34710
Ad-Aware 0.05% 0.00% 16 0 130420 34727
Babable 0.01% 0.00% 3 0 130420 34740
TotalGoodware 130420
TotalMalware 34743
TotalSample 165163

Please send an email to lxu@trustlook.com if you have any comments. Thanks.