VirusTotal APK Malware Detection Data - Week 29: 202100712-20210718

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210712_20210718.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
Fortinet 99.74% 0.01% 4178 2 30850 11
ESET-NOD32 99.69% 0.01% 4176 3 30849 13
Avira 99.43% 0.00% 4165 1 30851 24
K7GW 99.38% 0.06% 4163 17 30835 26
DrWeb 99.31% 0.08% 4160 24 30828 29
Trustlook 99.12% 0.17% 4152 52 30800 37
Ikarus 98.26% 0.01% 4116 4 30848 73
Kaspersky 97.18% 0.01% 4071 2 30850 118
Avast-Mobile 97.14% 0.27% 4069 84 30768 120
AhnLab-V3 95.46% 0.03% 3999 10 30842 190
McAfee 91.55% 0.01% 3835 2 30850 354
CAT-QuickHeal 91.05% 0.00% 3814 1 30851 375
Microsoft 88.66% 0.00% 3714 1 30851 475
Sophos 87.11% 0.02% 3649 7 30845 540
Cyren 82.88% 0.02% 3472 5 30847 717
McAfee-GW-Edition 81.69% 0.00% 3422 1 30851 767
NANO-Antivirus 80.00% 0.01% 3351 2 30850 838
Antiy-AVL 70.83% 0.10% 2967 31 30821 1222
Symantec 54.69% 0.01% 2291 2 30850 1898
ZoneAlarm 53.35% 0.01% 2235 3 30849 1954
SymantecMobileInsight 46.93% 1.66% 1966 512 30340 2223
AVG 25.83% 0.03% 1082 9 30843 3107
Avast 25.81% 0.03% 1081 9 30843 3108
Alibaba 25.09% 0.01% 1051 2 30850 3138
MAX 23.56% 0.00% 987 0 30852 3202
Tencent 22.80% 0.03% 955 9 30843 3234
Comodo 21.20% 0.04% 888 11 30841 3301
Zillya 19.69% 0.07% 825 23 30829 3364
ClamAV 15.80% 0.06% 662 17 30835 3527
GData 9.95% 0.00% 417 0 30852 3772
Yandex 8.86% 0.01% 371 4 30848 3818
Kingsoft 8.04% 0.00% 337 1 30851 3852
BitDefender 7.50% 0.00% 314 0 30852 3875
F-Secure 7.47% 0.00% 313 0 30852 3876
Emsisoft 7.38% 0.00% 309 0 30852 3880
Jiangmin 6.83% 0.46% 286 142 30710 3903
MicroWorld-eScan 4.44% 0.00% 186 0 30852 4003
TrendMicro-HouseCall 4.44% 0.02% 186 7 30845 4003
TrendMicro 4.30% 0.04% 180 13 30839 4009
Arcabit 3.72% 0.00% 156 0 30852 4033
Rising 3.01% 0.01% 126 4 30848 4063
Ad-Aware 2.48% 0.00% 104 0 30852 4085
VBA32 2.12% 0.01% 89 3 30849 4100
Zoner 2.05% 0.00% 86 0 30852 4103
Qihoo-360 1.69% 0.06% 71 17 30835 4118
Panda 0.88% 0.00% 37 1 30851 4152
Baidu 0.29% 0.00% 12 1 30851 4177
Babable 0.19% 0.01% 8 2 30850 4181
ViRobot 0.14% 0.00% 6 0 30852 4183
SentinelOne 0.14% 0.00% 6 0 30852 4183
VIPRE 0.05% 0.00% 2 0 30852 4187
F-Prot 0.05% 0.00% 2 0 30852 4187
CMC 0.02% 0.00% 1 0 30852 4188
SUPERAntiSpyware 0.02% 0.00% 1 0 30852 4188
Bkav 0.00% 0.00% 0 0 30852 4189
TotalDefense 0.00% 0.00% 0 0 30852 4189
nProtect 0.00% 0.00% 0 0 30852 4189
ALYac 0.00% 0.00% 0 0 30852 4189
Malwarebytes 0.00% 0.00% 0 0 30852 4189
CrowdStrike 0.00% 0.00% 0 0 30852 4189
K7AntiVirus 0.00% 0.00% 0 0 30852 4189
TheHacker 0.00% 0.00% 0 0 30852 4189
eScan 0.00% 0.00% 0 0 30852 4189
Invincea 0.00% 0.00% 0 0 30852 4189
Endgame 0.00% 0.00% 0 0 30852 4189
Webroot 0.00% 0.00% 0 0 30852 4189
AegisLab 0.00% 0.00% 0 0 30852 4189
AVware 0.00% 0.00% 0 0 30852 4189
TotalGoodware 30852
TotalMalware 4189
TotalSample 35041

Please send an email to lxu@trustlook.com if you have any comments. Thanks.