March 15, 2021

VirusTotal APK Malware Detection Data - Week 11: 20210308-20210314

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210308_20210314.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.40% 0.05% 12454 31 62929 75
ESET-NOD32 99.39% 0.01% 12453 4 62956 76
Avira 98.80% 0.00% 12379 0 62960 150
Avast-Mobile 98.51% 0.61% 12342 385 62575 187
DrWeb 97.63% 0.14% 12232 87 62873 297
Fortinet 97.45% 0.01% 12210 5 62955 319
AhnLab-V3 97.02% 0.03% 12156 22 62938 373
CAT-QuickHeal 95.46% 0.01% 11960 6 62954 569
Kaspersky 94.21% 0.00% 11803 1 62959 726
Ikarus 92.96% 0.08% 11647 50 62910 882
McAfee 92.07% 0.01% 11536 4 62956 993
McAfee-GW-Edition 87.87% 0.01% 11009 5 62955 1520
Trustlook 86.80% 0.10% 10875 65 62895 1654
AegisLab 84.84% 0.16% 10629 101 62859 1900
Sophos 82.48% 0.01% 10334 8 62952 2195
ZoneAlarm 81.00% 0.00% 10149 1 62959 2380
Microsoft 79.09% 0.02% 9909 10 62950 2620
Qihoo-360 78.13% 0.00% 9789 3 62957 2740
SymantecMobileInsight 70.77% 2.01% 8867 1264 61696 3662
NANO-Antivirus 70.54% 0.02% 8838 13 62947 3691
Tencent 65.91% 0.05% 8258 29 62931 4271
Cyren 64.58% 0.00% 8091 2 62958 4438
Symantec 62.13% 0.01% 7784 5 62955 4745
Alibaba 59.98% 0.00% 7515 3 62957 5014
F-Secure 52.41% 0.00% 6567 0 62960 5962
MAX 50.94% 0.00% 6382 0 62960 6147
Antiy-AVL 42.35% 0.09% 5306 56 62904 7223
Avast 33.64% 0.03% 4215 20 62940 8314
AVG 33.64% 0.03% 4215 20 62940 8314
Comodo 30.94% 0.07% 3876 43 62917 8653
Zillya 29.28% 0.07% 3669 47 62913 8860
ClamAV 13.27% 0.15% 1663 92 62868 10866
Kingsoft 8.02% 0.01% 1005 5 62955 11524
GData 7.54% 0.00% 945 0 62960 11584
BitDefender 7.10% 0.00% 890 0 62960 11639
Emsisoft 6.99% 0.00% 876 0 62960 11653
Yandex 6.46% 0.00% 809 0 62960 11720
Arcabit 6.39% 0.00% 800 0 62960 11729
Zoner 6.15% 0.00% 770 3 62957 11759
Jiangmin 4.82% 0.08% 604 48 62912 11925
MicroWorld-eScan 4.14% 0.00% 519 0 62960 12010
Rising 3.76% 0.01% 471 8 62952 12058
Ad-Aware 2.16% 0.00% 270 0 62960 12259
TrendMicro-HouseCall 1.87% 0.03% 234 21 62939 12295
TrendMicro 1.80% 0.05% 225 29 62931 12304
VBA32 0.82% 0.01% 103 5 62955 12426
TotalDefense 0.53% 0.00% 66 0 62960 12463
Panda 0.50% 0.00% 63 0 62960 12466
Baidu 0.37% 0.01% 46 9 62951 12483
ViRobot 0.27% 0.00% 34 1 62959 12495
VIPRE 0.10% 0.00% 13 0 62960 12516
Babable 0.09% 0.00% 11 3 62957 12518
K7AntiVirus 0.07% 0.00% 9 0 62960 12520
Malwarebytes 0.05% 0.00% 6 3 62957 12523
SentinelOne 0.04% 0.00% 5 0 62960 12524
ALYac 0.03% 0.00% 4 0 62960 12525
F-Prot 0.01% 0.00% 1 0 62960 12528
Bkav 0.00% 0.00% 0 1 62959 12529
nProtect 0.00% 0.00% 0 0 62960 12529
CMC 0.00% 0.00% 0 0 62960 12529
CrowdStrike 0.00% 0.00% 0 0 62960 12529
TheHacker 0.00% 0.00% 0 0 62960 12529
eScan 0.00% 0.00% 0 0 62960 12529
SUPERAntiSpyware 0.00% 0.00% 0 0 62960 12529
Invincea 0.00% 0.00% 0 0 62960 12529
Endgame 0.00% 0.00% 0 0 62960 12529
Webroot 0.00% 0.00% 0 0 62960 12529
AVware 0.00% 0.00% 0 0 62960 12529
TotalGoodware 62960
TotalMalware 12529
TotalSample 75489

Please send an email to lxu@trustlook.com if you have any comments. Thanks.