July 13, 2021

VirusTotal APK Malware Detection Data - Week 28: 202100705-20210711

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210705_20210711.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.79% 0.01% 4270 4 29642 9
K7GW 99.42% 0.08% 4254 23 29623 25
Trustlook 98.83% 0.12% 4229 36 29610 50
Ikarus 98.62% 0.03% 4220 8 29638 59
Fortinet 97.92% 0.00% 4190 0 29646 89
DrWeb 97.48% 0.07% 4171 21 29625 108
McAfee 96.87% 0.00% 4145 0 29646 134
AhnLab-V3 96.80% 0.03% 4142 9 29637 137
Kaspersky 96.70% 0.00% 4138 0 29646 141
Avira 95.28% 0.00% 4077 0 29646 202
Avast-Mobile 93.48% 0.28% 4000 83 29563 279
CAT-QuickHeal 91.17% 0.00% 3901 0 29646 378
Sophos 88.39% 0.02% 3782 5 29641 497
Microsoft 87.43% 0.01% 3741 3 29643 538
McAfee-GW-Edition 87.31% 0.01% 3736 4 29642 543
NANO-Antivirus 83.73% 0.02% 3583 7 29639 696
Symantec 76.44% 0.00% 3271 1 29645 1008
Antiy-AVL 75.65% 0.09% 3237 26 29620 1042
Cyren 68.47% 0.03% 2930 9 29637 1349
SymantecMobileInsight 65.48% 1.78% 2802 529 29117 1477
ZoneAlarm 50.48% 0.00% 2160 1 29645 2119
Alibaba 49.38% 0.00% 2113 1 29645 2166
Tencent 47.23% 0.02% 2021 7 29639 2258
MAX 43.47% 0.00% 1860 0 29646 2419
AVG 36.99% 0.01% 1583 3 29643 2696
Avast 36.95% 0.01% 1581 3 29643 2698
Comodo 29.28% 0.05% 1253 14 29632 3026
Zillya 24.05% 0.09% 1029 28 29618 3250
ClamAV 16.76% 0.03% 717 9 29637 3562
Kingsoft 14.61% 0.00% 625 1 29645 3654
Jiangmin 10.75% 0.62% 460 184 29462 3819
F-Secure 7.92% 0.00% 339 0 29646 3940
TrendMicro-HouseCall 7.06% 0.01% 302 3 29643 3977
TrendMicro 6.75% 0.00% 289 1 29645 3990
GData 6.22% 0.00% 266 0 29646 4013
BitDefender 6.03% 0.00% 258 0 29646 4021
Emsisoft 5.98% 0.00% 256 0 29646 4023
Yandex 5.66% 0.00% 242 1 29645 4037
AegisLab 5.07% 0.00% 217 0 29646 4062
Zoner 4.09% 0.00% 175 0 29646 4104
Arcabit 3.67% 0.00% 157 1 29645 4122
MicroWorld-eScan 3.48% 0.00% 149 0 29646 4130
Qihoo-360 3.37% 0.05% 144 16 29630 4135
Ad-Aware 1.66% 0.00% 71 0 29646 4208
Rising 0.84% 0.00% 36 1 29645 4243
VBA32 0.49% 0.00% 21 0 29646 4258
Panda 0.44% 0.00% 19 0 29646 4260
SentinelOne 0.35% 0.00% 15 0 29646 4264
Malwarebytes 0.30% 0.00% 13 0 29646 4266
Baidu 0.28% 0.01% 12 3 29643 4267
ViRobot 0.21% 0.00% 9 0 29646 4270
Babable 0.14% 0.00% 6 1 29645 4273
ALYac 0.02% 0.00% 1 0 29646 4278
VIPRE 0.02% 0.00% 1 0 29646 4278
F-Prot 0.02% 0.00% 1 0 29646 4278
Bkav 0.00% 0.00% 0 0 29646 4279
TotalDefense 0.00% 0.00% 0 0 29646 4279
nProtect 0.00% 0.00% 0 0 29646 4279
CMC 0.00% 0.00% 0 0 29646 4279
CrowdStrike 0.00% 0.00% 0 0 29646 4279
K7AntiVirus 0.00% 0.00% 0 0 29646 4279
TheHacker 0.00% 0.00% 0 0 29646 4279
eScan 0.00% 0.00% 0 0 29646 4279
SUPERAntiSpyware 0.00% 0.00% 0 0 29646 4279
Invincea 0.00% 0.00% 0 0 29646 4279
Endgame 0.00% 0.00% 0 0 29646 4279
Webroot 0.00% 0.00% 0 0 29646 4279
AVware 0.00% 0.00% 0 0 29646 4279
TotalGoodware 29646
TotalMalware 4279
TotalSample 33925

Please send an email to lxu@trustlook.com if you have any comments. Thanks.