February 1, 2021

VirusTotal APK Malware Detection Data - Week 5: 20210125-20210131

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210125_20210131.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.66% 0.03% 30767 21 68508 105
K7GW 99.55% 0.12% 30733 85 68444 139
Trustlook 99.30% 0.23% 30657 155 68374 215
Avira 99.20% 0.00% 30625 1 68528 247
Fortinet 98.92% 0.01% 30538 7 68522 334
Avast-Mobile 98.83% 0.63% 30510 431 68098 362
AhnLab-V3 98.30% 0.02% 30346 17 68512 526
ZoneAlarm 97.83% 0.03% 30202 20 68509 670
DrWeb 97.70% 0.15% 30161 100 68429 711
Kaspersky 94.46% 0.00% 29163 3 68526 1709
Ikarus 93.60% 0.13% 28895 87 68442 1977
McAfee 91.41% 0.01% 28221 4 68525 2651
Sophos 90.07% 0.02% 27805 12 68517 3067
F-Secure 88.72% 0.00% 27391 2 68527 3481
McAfee-GW-Edition 85.68% 0.03% 26451 22 68507 4421
Microsoft 77.63% 0.02% 23966 17 68512 6906
AegisLab 73.06% 0.14% 22554 96 68433 8318
Qihoo-360 64.77% 0.03% 19995 21 68508 10877
Antiy-AVL 62.85% 0.47% 19402 320 68209 11470
Cyren 60.69% 0.00% 18737 2 68527 12135
CAT-QuickHeal 52.97% 0.01% 16352 5 68524 14520
NANO-Antivirus 41.10% 0.02% 12689 17 68512 18183
Symantec 31.59% 0.01% 9752 7 68522 21120
Avast 30.47% 0.03% 9408 21 68508 21464
AVG 30.47% 0.03% 9408 21 68508 21464
Tencent 30.22% 0.05% 9329 37 68492 21543
SymantecMobileInsight 29.42% 1.25% 9081 859 67670 21791
Alibaba 28.03% 0.03% 8654 20 68509 22218
Zillya 27.57% 0.07% 8512 50 68479 22360
Comodo 27.22% 0.11% 8403 78 68451 22469
MAX 18.99% 0.00% 5863 0 68529 25009
ClamAV 18.08% 0.32% 5583 222 68307 25289
Yandex 17.51% 0.01% 5406 10 68519 25466
Jiangmin 16.25% 0.51% 5016 352 68177 25856
Kingsoft 8.46% 0.01% 2613 8 68521 28259
TrendMicro 7.57% 0.03% 2337 20 68509 28535
Rising 6.87% 0.01% 2121 9 68520 28751
VBA32 6.59% 0.01% 2036 10 68519 28836
TrendMicro-HouseCall 5.42% 0.04% 1674 26 68503 29198
GData 5.35% 0.00% 1653 0 68529 29219
BitDefender 4.22% 0.00% 1304 0 68529 29568
Emsisoft 4.16% 0.00% 1284 0 68529 29588
Arcabit 4.04% 0.00% 1246 1 68528 29626
Zoner 3.05% 0.02% 943 15 68514 29929
MicroWorld-eScan 1.97% 0.00% 608 0 68529 30264
TotalDefense 1.46% 0.00% 451 0 68529 30421
Ad-Aware 1.26% 0.00% 389 0 68529 30483
Panda 1.21% 0.00% 375 0 68529 30497
Baidu 0.18% 0.02% 56 13 68516 30816
K7AntiVirus 0.13% 0.00% 39 0 68529 30833
ALYac 0.11% 0.00% 35 0 68529 30837
ViRobot 0.11% 0.00% 33 1 68528 30839
SentinelOne 0.04% 0.00% 13 0 68529 30859
VIPRE 0.02% 0.00% 7 0 68529 30865
Malwarebytes 0.01% 0.00% 2 2 68527 30870
SUPERAntiSpyware 0.01% 0.00% 2 0 68529 30870
CMC 0.00% 0.00% 1 0 68529 30871
Bkav 0.00% 0.01% 0 9 68520 30872
nProtect 0.00% 0.00% 0 0 68529 30872
CrowdStrike 0.00% 0.00% 0 0 68529 30872
TheHacker 0.00% 0.00% 0 0 68529 30872
eScan 0.00% 0.00% 0 0 68529 30872
Babable 0.00% 0.00% 0 0 68529 30872
Invincea 0.00% 0.00% 0 0 68529 30872
F-Prot 0.00% 0.00% 0 0 68529 30872
Endgame 0.00% 0.00% 0 0 68529 30872
Webroot 0.00% 0.00% 0 0 68529 30872
AVware 0.00% 0.00% 0 0 68529 30872
TotalGoodware 68529
TotalMalware 30872
TotalSample 99401

Please send an email to lxu@trustlook.com if you have any comments. Thanks.