March 22, 2021

VirusTotal APK Malware Detection Data - Week 12: 20210315-20210321

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210315_20210321.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.27% 0.05% 6009 43 81035 44
Trustlook 99.04% 0.17% 5995 140 80938 58
ESET-NOD32 98.91% 0.01% 5987 7 81071 66
Avast-Mobile 98.60% 0.31% 5968 251 80827 85
Avira 98.36% 0.00% 5954 3 81075 99
DrWeb 97.90% 0.11% 5926 86 80992 127
CAT-QuickHeal 97.57% 0.01% 5906 11 81067 147
Fortinet 97.16% 0.01% 5881 10 81068 172
AhnLab-V3 96.15% 0.02% 5820 16 81062 233
McAfee 95.08% 0.00% 5755 4 81074 298
Kaspersky 94.40% 0.00% 5714 0 81078 339
Ikarus 92.66% 0.07% 5609 58 81020 444
AegisLab 86.60% 0.33% 5242 266 80812 811
McAfee-GW-Edition 83.48% 0.01% 5053 12 81066 1000
Microsoft 82.80% 0.01% 5012 7 81071 1041
NANO-Antivirus 80.54% 0.02% 4875 18 81060 1178
Sophos 75.25% 0.01% 4555 8 81070 1498
Qihoo-360 72.82% 0.01% 4408 10 81068 1645
Cyren 66.35% 0.00% 4016 1 81077 2037
ZoneAlarm 59.57% 0.01% 3606 5 81073 2447
SymantecMobileInsight 59.56% 1.97% 3605 1594 79484 2448
Tencent 56.48% 0.03% 3419 21 81057 2634
Symantec 49.86% 0.10% 3018 85 80993 3035
Alibaba 45.00% 0.00% 2724 2 81076 3329
Avast 42.03% 0.01% 2544 8 81070 3509
AVG 42.03% 0.01% 2544 8 81070 3509
MAX 41.93% 0.00% 2538 0 81078 3515
Comodo 40.53% 0.05% 2453 42 81036 3600
Zillya 39.67% 0.09% 2401 69 81009 3652
ClamAV 22.90% 0.11% 1386 91 80987 4667
Jiangmin 15.94% 0.20% 965 161 80917 5088
GData 11.94% 0.00% 723 0 81078 5330
Kingsoft 11.86% 0.01% 718 7 81071 5335
BitDefender 11.35% 0.00% 687 0 81078 5366
Emsisoft 11.20% 0.00% 678 0 81078 5375
F-Secure 9.75% 0.00% 590 0 81078 5463
Yandex 8.41% 0.00% 509 1 81077 5544
Arcabit 7.76% 0.00% 470 0 81078 5583
Antiy-AVL 6.99% 0.02% 423 13 81065 5630
Rising 6.67% 0.01% 404 5 81073 5649
MicroWorld-eScan 6.41% 0.00% 388 0 81078 5665
Zoner 6.15% 0.01% 372 8 81070 5681
Ad-Aware 3.90% 0.00% 236 0 81078 5817
TrendMicro 2.40% 0.03% 145 26 81052 5908
TrendMicro-HouseCall 2.03% 0.01% 123 11 81067 5930
VBA32 1.83% 0.01% 111 9 81069 5942
Panda 0.99% 0.00% 60 1 81077 5993
TotalDefense 0.63% 0.00% 38 0 81078 6015
Baidu 0.58% 0.01% 35 5 81073 6018
VIPRE 0.50% 0.00% 30 0 81078 6023
ViRobot 0.30% 0.00% 18 0 81078 6035
Babable 0.23% 0.01% 14 11 81067 6039
SentinelOne 0.18% 0.00% 11 0 81078 6042
Malwarebytes 0.12% 0.00% 7 3 81075 6046
K7AntiVirus 0.12% 0.00% 7 0 81078 6046
ALYac 0.08% 0.00% 5 1 81077 6048
F-Prot 0.07% 0.00% 4 0 81078 6049
SUPERAntiSpyware 0.02% 0.00% 1 0 81078 6052
Bkav 0.00% 0.00% 0 0 81078 6053
nProtect 0.00% 0.00% 0 0 81078 6053
CMC 0.00% 0.00% 0 0 81078 6053
CrowdStrike 0.00% 0.00% 0 0 81078 6053
TheHacker 0.00% 0.00% 0 0 81078 6053
eScan 0.00% 0.00% 0 0 81078 6053
Invincea 0.00% 0.00% 0 0 81078 6053
Endgame 0.00% 0.00% 0 0 81078 6053
Webroot 0.00% 0.00% 0 0 81078 6053
AVware 0.00% 0.00% 0 0 81078 6053
TotalGoodware 81078
TotalMalware 6053
TotalSample 87131

Please send an email to lxu@trustlook.com if you have any comments. Thanks.