April 20, 2021

VirusTotal APK Malware Detection Data - Week 16: 202100412-20210418

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210412_20210418.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
K7GW 99.82% 0.16% 48196 266 169910 88
ESET-NOD32 99.43% 0.01% 48008 18 170158 276
Avira 99.16% 0.00% 47880 5 170171 404
Trustlook 98.91% 0.04% 47759 61 170115 525
Fortinet 98.78% 0.01% 47694 18 170158 590
CAT-QuickHeal 98.74% 0.01% 47677 20 170156 607
AhnLab-V3 97.96% 0.03% 47298 48 170128 986
Avast-Mobile 96.50% 0.33% 46596 561 169615 1688
SymantecMobileInsight 96.30% 3.33% 46496 5673 164503 1788
AegisLab 95.79% 0.09% 46249 151 170025 2035
Symantec 94.93% 0.13% 45836 225 169951 2448
Alibaba 94.77% 0.00% 45760 6 170170 2524
McAfee-GW-Edition 92.51% 0.08% 44667 138 170038 3617
Tencent 91.58% 0.11% 44220 193 169983 4064
DrWeb 90.43% 0.40% 43664 685 169491 4620
McAfee 89.81% 0.01% 43363 11 170165 4921
NANO-Antivirus 87.72% 0.05% 42354 83 170093 5930
Qihoo-360 86.18% 0.04% 41612 65 170111 6672
Sophos 79.80% 0.01% 38532 17 170159 9752
Ikarus 77.93% 0.13% 37627 223 169953 10657
Kaspersky 77.71% 0.00% 37522 0 170176 10762
Microsoft 77.02% 0.01% 37187 16 170160 11097
Cyren 69.76% 0.01% 33681 11 170165 14603
MAX 67.59% 0.00% 32637 0 170176 15647
ZoneAlarm 57.17% 0.00% 27605 1 170175 20679
Comodo 44.99% 0.07% 21725 120 170056 26559
AVG 43.42% 0.01% 20966 14 170162 27318
Avast 43.06% 0.01% 20789 14 170162 27495
Zillya 28.09% 0.09% 13564 153 170023 34720
ClamAV 24.04% 0.12% 11609 206 169970 36675
Jiangmin 23.44% 0.57% 11316 978 169198 36968
GData 19.22% 0.00% 9280 0 170176 39004
BitDefender 16.00% 0.00% 7727 0 170176 40557
Emsisoft 15.80% 0.00% 7631 0 170176 40653
Rising 15.71% 0.01% 7586 15 170161 40698
Kingsoft 12.12% 0.02% 5853 35 170141 42431
VBA32 9.14% 0.01% 4414 17 170159 43870
Zoner 8.07% 0.00% 3896 3 170173 44388
MicroWorld-eScan 6.60% 0.00% 3185 0 170176 45099
Antiy-AVL 6.12% 0.04% 2953 70 170106 45331
Yandex 5.94% 0.00% 2866 1 170175 45418
TrendMicro-HouseCall 5.38% 0.00% 2598 4 170172 45686
Arcabit 5.06% 0.00% 2441 0 170176 45843
TrendMicro 4.88% 0.01% 2358 14 170162 45926
F-Secure 3.17% 0.00% 1531 2 170174 46753
Ad-Aware 1.64% 0.00% 791 0 170176 47493
Panda 1.10% 0.00% 530 6 170170 47754
Babable 0.27% 0.03% 130 55 170121 48154
K7AntiVirus 0.19% 0.00% 90 0 170176 48194
ALYac 0.12% 0.00% 58 0 170176 48226
Baidu 0.12% 0.01% 56 10 170166 48228
ViRobot 0.11% 0.00% 55 3 170173 48229
VIPRE 0.06% 0.00% 31 0 170176 48253
F-Prot 0.05% 0.00% 26 0 170176 48258
SentinelOne 0.04% 0.00% 17 1 170175 48267
Malwarebytes 0.03% 0.01% 14 10 170166 48270
CMC 0.01% 0.00% 3 0 170176 48281
SUPERAntiSpyware 0.01% 0.00% 3 0 170176 48281
Bkav 0.00% 0.00% 0 1 170175 48284
TotalDefense 0.00% 0.00% 0 0 170176 48284
nProtect 0.00% 0.00% 0 0 170176 48284
CrowdStrike 0.00% 0.00% 0 0 170176 48284
TheHacker 0.00% 0.00% 0 0 170176 48284
eScan 0.00% 0.00% 0 0 170176 48284
Invincea 0.00% 0.00% 0 0 170176 48284
Endgame 0.00% 0.00% 0 0 170176 48284
Webroot 0.00% 0.00% 0 0 170176 48284
AVware 0.00% 0.00% 0 0 170176 48284
TotalGoodware 170176
TotalMalware 48284
TotalSample 218460

Please send an email to lxu@trustlook.com if you have any comments. Thanks.