April 5, 2021

VirusTotal APK Malware Detection Data - Week 14: 20210329-20210404

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210329_20210404.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
K7GW 99.88% 0.18% 54389 287 158139 66
ESET-NOD32 99.49% 0.02% 54180 26 158400 275
Fortinet 99.12% 0.02% 53977 26 158400 478
Avira 99.04% 0.00% 53934 2 158424 521
CAT-QuickHeal 98.71% 0.01% 53755 16 158410 700
AhnLab-V3 97.94% 0.02% 53333 39 158387 1122
Trustlook 97.92% 0.05% 53323 74 158352 1132
Avast-Mobile 97.75% 0.53% 53228 839 157587 1227
SymantecMobileInsight 95.73% 3.67% 52128 5822 152604 2327
AegisLab 95.22% 0.18% 51851 291 158135 2604
McAfee 94.56% 0.01% 51494 12 158414 2961
Alibaba 94.05% 0.01% 51214 8 158418 3241
McAfee-GW-Edition 93.60% 0.01% 50971 15 158411 3484
Symantec 93.28% 0.13% 50794 199 158227 3661
DrWeb 93.11% 0.49% 50701 777 157649 3754
Qihoo-360 92.53% 0.02% 50388 34 158392 4067
Tencent 92.47% 0.07% 50357 117 158309 4098
Kaspersky 84.75% 0.00% 46150 3 158423 8305
NANO-Antivirus 84.23% 0.05% 45867 76 158350 8588
Ikarus 83.92% 0.11% 45701 179 158247 8754
Microsoft 80.72% 0.01% 43954 17 158409 10501
Sophos 80.46% 0.02% 43816 25 158401 10639
MAX 68.65% 0.00% 37384 1 158425 17071
Cyren 67.86% 0.01% 36955 9 158417 17500
ZoneAlarm 59.71% 0.01% 32515 9 158417 21940
Avast 51.59% 0.01% 28092 16 158410 26363
AVG 51.59% 0.01% 28092 16 158410 26363
Comodo 45.26% 0.07% 24648 107 158319 29807
Zillya 28.58% 0.10% 15563 154 158272 38892
Jiangmin 27.41% 0.34% 14924 535 157891 39531
ClamAV 23.42% 0.15% 12756 232 158194 41699
GData 18.06% 0.00% 9835 1 158425 44620
Rising 15.95% 0.01% 8685 13 158413 45770
BitDefender 14.53% 0.00% 7915 0 158426 46540
Emsisoft 14.41% 0.00% 7847 0 158426 46608
TrendMicro 12.13% 0.02% 6606 31 158395 47849
TrendMicro-HouseCall 12.09% 0.01% 6583 12 158414 47872
VBA32 12.08% 0.01% 6576 17 158409 47879
Kingsoft 11.79% 0.01% 6419 20 158406 48036
Antiy-AVL 10.68% 0.05% 5818 84 158342 48637
Zoner 9.10% 0.00% 4954 5 158421 49501
MicroWorld-eScan 7.05% 0.00% 3839 0 158426 50616
Arcabit 6.70% 0.00% 3649 0 158426 50806
Yandex 5.59% 0.00% 3046 2 158424 51409
F-Secure 3.63% 0.00% 1974 1 158425 52481
Ad-Aware 1.91% 0.00% 1039 0 158426 53416
Panda 1.27% 0.00% 691 4 158422 53764
K7AntiVirus 0.33% 0.00% 181 0 158426 54274
Babable 0.32% 0.03% 174 48 158378 54281
ALYac 0.25% 0.00% 137 0 158426 54318
F-Prot 0.12% 0.00% 68 0 158426 54387
Baidu 0.11% 0.01% 59 16 158410 54396
VIPRE 0.09% 0.00% 47 0 158426 54408
ViRobot 0.08% 0.00% 45 2 158424 54410
SentinelOne 0.04% 0.00% 23 1 158425 54432
TotalDefense 0.01% 0.00% 6 0 158426 54449
SUPERAntiSpyware 0.01% 0.00% 6 0 158426 54449
Malwarebytes 0.01% 0.01% 4 8 158418 54451
CMC 0.00% 0.00% 1 0 158426 54454
Bkav 0.00% 0.00% 0 0 158426 54455
nProtect 0.00% 0.00% 0 0 158426 54455
CrowdStrike 0.00% 0.00% 0 0 158426 54455
TheHacker 0.00% 0.00% 0 0 158426 54455
eScan 0.00% 0.00% 0 0 158426 54455
Invincea 0.00% 0.00% 0 0 158426 54455
Endgame 0.00% 0.00% 0 0 158426 54455
Webroot 0.00% 0.00% 0 0 158426 54455
AVware 0.00% 0.00% 0 0 158426 54455
TotalGoodware 158426
TotalMalware 54455
TotalSample 212881

Please send an email to lxu@trustlook.com if you have any comments. Thanks.