March 30, 2021

VirusTotal APK Malware Detection Data - Week 13: 20210322-20210328

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210322_20210328.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
K7GW 99.51% 0.05% 18101 57 106465 90
ESET-NOD32 99.31% 0.01% 18065 11 106511 126
Avast-Mobile 99.14% 0.44% 18034 470 106052 157
DrWeb 98.97% 0.10% 18003 109 106413 188
Fortinet 98.57% 0.02% 17931 16 106506 260
CAT-QuickHeal 97.30% 0.01% 17699 9 106513 492
Avira 95.49% 0.00% 17371 2 106520 820
AhnLab-V3 95.18% 0.03% 17314 27 106495 877
McAfee 90.51% 0.00% 16465 5 106517 1726
Ikarus 88.33% 0.04% 16069 46 106476 2122
Kaspersky 86.77% 0.00% 15785 1 106521 2406
McAfee-GW-Edition 82.38% 0.01% 14985 6 106516 3206
Microsoft 76.87% 0.01% 13984 11 106511 4207
AegisLab 76.17% 0.13% 13856 141 106381 4335
Qihoo-360 73.64% 0.01% 13396 15 106507 4795
SymantecMobileInsight 73.14% 2.16% 13304 2296 104226 4887
NANO-Antivirus 71.49% 0.02% 13005 19 106503 5186
Cyren 70.97% 0.00% 12911 3 106519 5280
Sophos 70.33% 0.02% 12793 16 106506 5398
Tencent 68.73% 0.02% 12502 18 106504 5689
Trustlook 63.57% 0.04% 11564 45 106477 6627
Symantec 60.62% 0.15% 11028 163 106359 7163
Alibaba 57.80% 0.00% 10514 2 106520 7677
ZoneAlarm 57.51% 0.01% 10461 6 106516 7730
MAX 43.70% 0.00% 7949 0 106522 10242
Avast 42.62% 0.02% 7753 19 106503 10438
AVG 42.62% 0.02% 7753 19 106503 10438
Comodo 34.87% 0.07% 6344 75 106447 11847
Zillya 25.56% 0.12% 4649 130 106392 13542
GData 18.01% 0.00% 3276 1 106521 14915
Jiangmin 17.86% 0.28% 3249 293 106229 14942
BitDefender 16.59% 0.00% 3018 0 106522 15173
Emsisoft 16.38% 0.00% 2980 0 106522 15211
ClamAV 16.30% 0.12% 2966 129 106393 15225
MicroWorld-eScan 11.04% 0.00% 2009 0 106522 16182
Rising 10.32% 0.00% 1877 2 106520 16314
Arcabit 9.50% 0.00% 1729 1 106521 16462
Ad-Aware 7.91% 0.00% 1439 0 106522 16752
Kingsoft 6.62% 0.01% 1205 10 106512 16986
Yandex 6.03% 0.00% 1097 2 106520 17094
Zoner 5.99% 0.01% 1089 7 106515 17102
F-Secure 5.60% 0.00% 1018 0 106522 17173
Antiy-AVL 4.08% 0.03% 742 32 106490 17449
VBA32 4.03% 0.00% 734 5 106517 17457
TrendMicro-HouseCall 3.97% 0.01% 723 10 106512 17468
TrendMicro 3.34% 0.02% 608 18 106504 17583
TotalDefense 1.46% 0.00% 266 0 106522 17925
Panda 0.74% 0.00% 135 5 106517 18056
Baidu 0.22% 0.01% 40 12 106510 18151
Babable 0.21% 0.03% 38 30 106492 18153
ViRobot 0.20% 0.00% 36 1 106521 18155
K7AntiVirus 0.15% 0.00% 27 0 106522 18164
ALYac 0.14% 0.00% 25 2 106520 18166
VIPRE 0.10% 0.00% 19 0 106522 18172
SentinelOne 0.05% 0.00% 10 1 106521 18181
F-Prot 0.03% 0.00% 5 0 106522 18186
SUPERAntiSpyware 0.01% 0.00% 2 0 106522 18189
Malwarebytes 0.01% 0.00% 1 0 106522 18190
Bkav 0.00% 0.00% 0 0 106522 18191
nProtect 0.00% 0.00% 0 0 106522 18191
CMC 0.00% 0.00% 0 0 106522 18191
CrowdStrike 0.00% 0.00% 0 0 106522 18191
TheHacker 0.00% 0.00% 0 0 106522 18191
eScan 0.00% 0.00% 0 0 106522 18191
Invincea 0.00% 0.00% 0 0 106522 18191
Endgame 0.00% 0.00% 0 0 106522 18191
Webroot 0.00% 0.00% 0 0 106522 18191
AVware 0.00% 0.00% 0 0 106522 18191
TotalGoodware 106522
TotalMalware 18191
TotalSample 124713

Please send an email to lxu@trustlook.com if you have any comments. Thanks.