March 8, 2021

VirusTotal APK Malware Detection Data - Week 10: 20210301-20210307

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210301_20210307.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.74% 0.04% 17872 22 55662 46
ESET-NOD32 99.22% 0.01% 17779 5 55679 139
Avast-Mobile 99.06% 0.30% 17750 167 55517 168
CAT-QuickHeal 98.79% 0.01% 17702 4 55680 216
AhnLab-V3 98.75% 0.03% 17694 15 55669 224
Trustlook 98.69% 0.15% 17683 84 55600 235
ZoneAlarm 98.67% 0.01% 17679 4 55680 239
Avira 98.06% 0.00% 17570 1 55683 348
Fortinet 97.98% 0.01% 17556 7 55677 362
McAfee 96.74% 0.01% 17333 3 55681 585
Kaspersky 96.60% 0.00% 17308 2 55682 610
Ikarus 95.69% 0.08% 17145 42 55642 773
DrWeb 95.48% 0.09% 17108 52 55632 810
McAfee-GW-Edition 95.07% 0.01% 17034 3 55681 884
F-Secure 92.65% 0.00% 16601 1 55683 1317
Qihoo-360 92.07% 0.02% 16497 9 55675 1421
Sophos 92.00% 0.01% 16485 4 55680 1433
NANO-Antivirus 91.29% 0.01% 16358 6 55678 1560
SymantecMobileInsight 88.32% 1.82% 15825 1015 54669 2093
Microsoft 87.81% 0.02% 15734 12 55672 2184
Symantec 85.16% 0.00% 15259 2 55682 2659
Alibaba 83.85% 0.01% 15025 3 55681 2893
Tencent 81.34% 0.04% 14575 25 55659 3343
AegisLab 81.21% 0.10% 14551 54 55630 3367
Avast 80.87% 0.03% 14490 19 55665 3428
AVG 80.87% 0.03% 14490 19 55665 3428
Cyren 77.78% 0.00% 13936 1 55683 3982
Antiy-AVL 74.86% 0.12% 13413 67 55617 4505
Comodo 58.67% 0.06% 10513 36 55648 7405
MAX 57.52% 0.00% 10307 1 55683 7611
Zillya 38.65% 0.07% 6926 41 55643 10992
ClamAV 27.58% 0.17% 4942 96 55588 12976
Rising 18.36% 0.01% 3290 4 55680 14628
GData 17.90% 0.00% 3207 0 55684 14711
Jiangmin 13.99% 0.27% 2506 150 55534 15412
BitDefender 11.80% 0.00% 2114 0 55684 15804
TrendMicro 11.78% 0.04% 2111 24 55660 15807
Arcabit 11.73% 0.00% 2101 0 55684 15817
TrendMicro-HouseCall 11.68% 0.04% 2092 23 55661 15826
Emsisoft 11.67% 0.00% 2091 0 55684 15827
Kingsoft 11.31% 0.00% 2027 1 55683 15891
Zoner 9.87% 0.01% 1769 5 55679 16149
Yandex 8.41% 0.00% 1507 1 55683 16411
MicroWorld-eScan 6.49% 0.00% 1163 0 55684 16755
VBA32 5.56% 0.01% 996 6 55678 16922
Ad-Aware 3.31% 0.00% 593 0 55684 17325
Panda 2.08% 0.00% 372 0 55684 17546
K7AntiVirus 0.84% 0.00% 150 0 55684 17768
Baidu 0.58% 0.02% 104 9 55675 17814
VIPRE 0.33% 0.00% 59 0 55684 17859
ViRobot 0.25% 0.00% 45 1 55683 17873
TotalDefense 0.18% 0.00% 33 0 55684 17885
Malwarebytes 0.04% 0.00% 8 1 55683 17910
SentinelOne 0.04% 0.00% 8 0 55684 17910
ALYac 0.02% 0.00% 3 0 55684 17915
Bkav 0.00% 0.00% 0 0 55684 17918
nProtect 0.00% 0.00% 0 0 55684 17918
CMC 0.00% 0.00% 0 0 55684 17918
CrowdStrike 0.00% 0.00% 0 0 55684 17918
TheHacker 0.00% 0.00% 0 0 55684 17918
eScan 0.00% 0.00% 0 0 55684 17918
Babable 0.00% 0.00% 0 0 55684 17918
SUPERAntiSpyware 0.00% 0.00% 0 0 55684 17918
Invincea 0.00% 0.00% 0 0 55684 17918
F-Prot 0.00% 0.00% 0 0 55684 17918
Endgame 0.00% 0.00% 0 0 55684 17918
Webroot 0.00% 0.00% 0 0 55684 17918
AVware 0.00% 0.00% 0 0 55684 17918
TotalGoodware 55684
TotalMalware 17918
TotalSample 73602

Please send an email to lxu@trustlook.com if you have any comments. Thanks.