June 1, 2021

VirusTotal APK Malware Detection Data - Week 22: 202100524-20210530

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210514_20210530.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.66% 0.01% 21031 10 84021 71
Trustlook 99.30% 0.17% 20954 145 83886 148
Avira 99.23% 0.00% 20940 0 84031 162
Avast-Mobile 99.18% 0.33% 20928 279 83752 174
Fortinet 98.96% 0.01% 20883 7 84024 219
Ikarus 98.67% 0.06% 20821 51 83980 281
AhnLab-V3 98.64% 0.04% 20814 34 83997 288
Kaspersky 97.88% 0.00% 20654 2 84029 448
CAT-QuickHeal 97.05% 0.01% 20479 7 84024 623
K7GW 95.21% 0.10% 20092 82 83949 1010
DrWeb 91.43% 0.15% 19293 122 83909 1809
Sophos 90.49% 0.01% 19095 9 84022 2007
Microsoft 82.01% 0.02% 17305 19 84012 3797
McAfee 81.43% 0.00% 17183 2 84029 3919
McAfee-GW-Edition 76.63% 0.00% 16170 4 84027 4932
AegisLab 71.22% 0.10% 15028 83 83948 6074
ZoneAlarm 66.81% 0.00% 14099 3 84028 7003
Cyren 58.04% 0.00% 12248 2 84029 8854
AVG 56.56% 0.00% 11935 4 84027 9167
SymantecMobileInsight 56.39% 2.32% 11900 1947 82084 9202
Avast 56.26% 0.00% 11872 4 84027 9230
NANO-Antivirus 54.87% 0.05% 11579 43 83988 9523
Tencent 47.45% 0.05% 10013 40 83991 11089
Symantec 45.76% 0.01% 9657 12 84019 11445
Alibaba 38.22% 0.01% 8066 7 84024 13036
ClamAV 30.34% 0.12% 6403 103 83928 14699
MAX 27.94% 0.00% 5896 0 84031 15206
Zillya 21.46% 0.11% 4529 95 83936 16573
Comodo 18.61% 0.08% 3928 69 83962 17174
Kingsoft 17.23% 0.01% 3635 6 84025 17467
GData 10.34% 0.00% 2181 0 84031 18921
Rising 9.36% 0.02% 1976 18 84013 19126
Jiangmin 8.72% 0.80% 1840 675 83356 19262
TrendMicro-HouseCall 7.19% 0.02% 1518 15 84016 19584
F-Secure 7.12% 0.00% 1502 0 84031 19600
TrendMicro 6.95% 0.04% 1466 31 84000 19636
BitDefender 6.87% 0.00% 1450 0 84031 19652
Emsisoft 6.73% 0.00% 1420 0 84031 19682
Zoner 6.12% 0.02% 1291 20 84011 19811
Yandex 5.24% 0.00% 1106 2 84029 19996
Arcabit 3.54% 0.00% 746 0 84031 20356
VBA32 3.21% 0.01% 677 8 84023 20425
MicroWorld-eScan 2.89% 0.00% 610 0 84031 20492
VIPRE 1.42% 0.00% 299 0 84031 20803
Antiy-AVL 1.39% 0.01% 293 10 84021 20809
Panda 1.12% 0.00% 236 0 84031 20866
Ad-Aware 0.61% 0.00% 128 0 84031 20974
Babable 0.48% 0.03% 102 24 84007 21000
F-Prot 0.16% 0.00% 33 0 84031 21069
Baidu 0.14% 0.01% 29 11 84020 21073
ViRobot 0.07% 0.00% 14 0 84031 21088
SentinelOne 0.03% 0.00% 7 0 84031 21095
K7AntiVirus 0.02% 0.00% 4 0 84031 21098
SUPERAntiSpyware 0.02% 0.00% 4 0 84031 21098
ALYac 0.01% 0.00% 2 0 84031 21100
Malwarebytes 0.00% 0.00% 1 0 84031 21101
Bkav 0.00% 0.00% 0 1 84030 21102
TotalDefense 0.00% 0.00% 0 0 84031 21102
nProtect 0.00% 0.00% 0 0 84031 21102
CMC 0.00% 0.00% 0 0 84031 21102
CrowdStrike 0.00% 0.00% 0 0 84031 21102
TheHacker 0.00% 0.00% 0 0 84031 21102
eScan 0.00% 0.00% 0 0 84031 21102
Invincea 0.00% 0.00% 0 0 84031 21102
Endgame 0.00% 0.00% 0 0 84031 21102
Webroot 0.00% 0.00% 0 0 84031 21102
AVware 0.00% 0.00% 0 0 84031 21102
Qihoo-360 0.00% 0.00% 0 0 84031 21102
TotalGoodware 84031
TotalMalware 21102
TotalSample 105133

Please send an email to lxu@trustlook.com if you have any comments. Thanks.