VirusTotal APK Malware Detection Data - Week 26: 202100621-20210627

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210621_20210627.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
K7GW 99.86% 0.06% 12274 16 26212 17
Avira 99.85% 0.00% 12272 0 26228 19
Fortinet 99.77% 0.00% 12263 1 26227 28
ESET-NOD32 99.72% 0.02% 12257 5 26223 34
AhnLab-V3 99.63% 0.05% 12245 12 26216 46
Trustlook 99.61% 0.11% 12243 30 26198 48
Avast-Mobile 99.60% 0.44% 12242 116 26112 49
Ikarus 99.22% 0.05% 12195 13 26215 96
DrWeb 97.91% 0.08% 12034 20 26208 257
Kaspersky 97.45% 0.00% 11977 0 26228 314
CAT-QuickHeal 97.19% 0.00% 11946 0 26228 345
Sophos 92.55% 0.00% 11375 1 26227 916
McAfee 84.49% 0.00% 10385 0 26228 1906
McAfee-GW-Edition 79.82% 0.00% 9811 0 26228 2480
Cyren 77.22% 0.03% 9491 8 26220 2800
AegisLab 74.45% 0.05% 9151 14 26214 3140
ZoneAlarm 74.44% 0.00% 9149 1 26227 3142
Microsoft 72.37% 0.01% 8895 2 26226 3396
NANO-Antivirus 63.41% 0.02% 7794 5 26223 4497
Zillya 42.41% 0.07% 5213 18 26210 7078
Antiy-AVL 40.54% 0.06% 4983 16 26212 7308
SymantecMobileInsight 39.32% 1.87% 4833 491 25737 7458
Symantec 38.35% 0.00% 4713 1 26227 7578
Tencent 36.13% 0.03% 4441 9 26219 7850
Alibaba 35.87% 0.01% 4409 2 26226 7882
AVG 26.66% 0.00% 3277 1 26227 9014
Avast 26.47% 0.00% 3254 1 26227 9037
Kingsoft 24.95% 0.00% 3067 1 26227 9224
MAX 24.79% 0.00% 3047 0 26228 9244
ClamAV 14.94% 0.03% 1836 9 26219 10455
F-Secure 7.62% 0.00% 937 0 26228 11354
Zoner 7.04% 0.00% 865 1 26227 11426
Comodo 3.98% 0.03% 489 7 26221 11802
Jiangmin 3.27% 0.80% 402 209 26019 11889
Yandex 2.40% 0.00% 295 0 26228 11996
GData 2.12% 0.00% 260 0 26228 12031
BitDefender 2.07% 0.00% 255 0 26228 12036
Emsisoft 2.04% 0.00% 251 0 26228 12040
MicroWorld-eScan 1.50% 0.00% 184 0 26228 12107
Arcabit 1.36% 0.00% 167 0 26228 12124
TrendMicro-HouseCall 0.94% 0.01% 115 3 26225 12176
TrendMicro 0.86% 0.02% 106 4 26224 12185
Ad-Aware 0.72% 0.00% 88 0 26228 12203
Rising 0.33% 0.00% 40 0 26228 12251
Babable 0.26% 0.02% 32 6 26222 12259
VBA32 0.23% 0.01% 28 3 26225 12263
Panda 0.15% 0.00% 18 0 26228 12273
Baidu 0.11% 0.01% 13 3 26225 12278
ViRobot 0.10% 0.00% 12 0 26228 12279
SentinelOne 0.07% 0.00% 9 1 26227 12282
F-Prot 0.02% 0.00% 2 0 26228 12289
CMC 0.01% 0.00% 1 0 26228 12290
ALYac 0.01% 0.00% 1 0 26228 12290
SUPERAntiSpyware 0.01% 0.00% 1 0 26228 12290
VIPRE 0.01% 0.00% 1 0 26228 12290
Bkav 0.00% 0.00% 0 0 26228 12291
TotalDefense 0.00% 0.00% 0 0 26228 12291
nProtect 0.00% 0.00% 0 0 26228 12291
Malwarebytes 0.00% 0.00% 0 0 26228 12291
CrowdStrike 0.00% 0.00% 0 0 26228 12291
K7AntiVirus 0.00% 0.00% 0 0 26228 12291
TheHacker 0.00% 0.00% 0 0 26228 12291
eScan 0.00% 0.00% 0 0 26228 12291
Invincea 0.00% 0.00% 0 0 26228 12291
Endgame 0.00% 0.00% 0 0 26228 12291
Webroot 0.00% 0.00% 0 0 26228 12291
AVware 0.00% 0.00% 0 0 26228 12291
Qihoo-360 0.00% 0.00% 0 0 26228 12291
TotalGoodware 26228
TotalMalware 12291
TotalSample 38519

Please send an email to lxu@trustlook.com if you have any comments. Thanks.