February 16, 2021

VirusTotal APK Malware Detection Data - Week 7: 20210208-20210214

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210208_20210214.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.51% 0.08% 18695 39 49665 93
Trustlook 99.48% 0.16% 18690 81 49623 98
ESET-NOD32 99.35% 0.02% 18666 12 49692 122
Avira 98.71% 0.00% 18546 0 49704 242
Avast-Mobile 98.01% 0.46% 18414 229 49475 374
AhnLab-V3 97.99% 0.03% 18410 16 49688 378
Fortinet 97.14% 0.01% 18251 4 49700 537
Ikarus 96.30% 0.11% 18093 56 49648 695
DrWeb 94.39% 0.13% 17734 66 49638 1054
McAfee 94.13% 0.00% 17685 1 49703 1103
F-Secure 93.62% 0.00% 17589 0 49704 1199
McAfee-GW-Edition 91.60% 0.01% 17210 4 49700 1578
ZoneAlarm 87.62% 0.00% 16462 0 49704 2326
Microsoft 86.68% 0.02% 16285 10 49694 2503
Kaspersky 86.25% 0.00% 16204 0 49704 2584
CAT-QuickHeal 83.05% 0.01% 15603 7 49697 3185
AegisLab 82.63% 0.20% 15525 98 49606 3263
Sophos 73.31% 0.01% 13773 6 49698 5015
SymantecMobileInsight 73.12% 2.17% 13738 1079 48625 5050
NANO-Antivirus 71.60% 0.03% 13453 13 49691 5335
Cyren 71.00% 0.00% 13339 0 49704 5449
Avast 66.08% 0.03% 12415 16 49688 6373
AVG 66.08% 0.03% 12415 16 49688 6373
Tencent 64.12% 0.05% 12047 26 49678 6741
Antiy-AVL 63.26% 0.18% 11886 91 49613 6902
Symantec 61.26% 0.02% 11509 10 49694 7279
Qihoo-360 58.92% 0.01% 11070 7 49697 7718
Alibaba 51.24% 0.01% 9627 6 49698 9161
Comodo 35.09% 0.09% 6592 46 49658 12196
MAX 34.22% 0.00% 6430 2 49702 12358
Zillya 22.04% 0.05% 4140 27 49677 14648
ClamAV 18.99% 0.20% 3568 99 49605 15220
Jiangmin 18.38% 0.28% 3453 141 49563 15335
GData 13.01% 0.00% 2445 0 49704 16343
Yandex 12.16% 0.01% 2284 7 49697 16504
Zoner 12.07% 0.01% 2268 5 49699 16520
Rising 10.61% 0.00% 1993 1 49703 16795
BitDefender 8.61% 0.00% 1618 0 49704 17170
Emsisoft 8.50% 0.00% 1597 0 49704 17191
Arcabit 8.50% 0.00% 1597 1 49703 17191
TrendMicro 8.43% 0.05% 1584 25 49679 17204
TrendMicro-HouseCall 7.50% 0.04% 1409 20 49684 17379
Kingsoft 6.79% 0.01% 1276 4 49700 17512
MicroWorld-eScan 4.74% 0.00% 891 0 49704 17897
VBA32 4.60% 0.02% 865 10 49694 17923
Ad-Aware 2.97% 0.00% 558 0 49704 18230
VIPRE 2.20% 0.00% 414 0 49704 18374
Panda 1.37% 0.00% 258 0 49704 18530
TotalDefense 0.59% 0.00% 111 0 49704 18677
Baidu 0.17% 0.01% 32 7 49697 18756
ViRobot 0.11% 0.00% 20 0 49704 18768
K7AntiVirus 0.05% 0.00% 10 0 49704 18778
SentinelOne 0.03% 0.00% 5 0 49704 18783
ALYac 0.02% 0.00% 3 0 49704 18785
Malwarebytes 0.01% 0.00% 1 1 49703 18787
Bkav 0.00% 0.00% 0 2 49702 18788
nProtect 0.00% 0.00% 0 0 49704 18788
CMC 0.00% 0.00% 0 0 49704 18788
CrowdStrike 0.00% 0.00% 0 0 49704 18788
TheHacker 0.00% 0.00% 0 0 49704 18788
eScan 0.00% 0.00% 0 0 49704 18788
Babable 0.00% 0.00% 0 0 49704 18788
SUPERAntiSpyware 0.00% 0.00% 0 0 49704 18788
Invincea 0.00% 0.00% 0 0 49704 18788
F-Prot 0.00% 0.00% 0 0 49704 18788
Endgame 0.00% 0.00% 0 0 49704 18788
Webroot 0.00% 0.00% 0 0 49704 18788
AVware 0.00% 0.00% 0 0 49704 18788
TotalGoodware 49704
TotalMalware 18788
TotalSample 68492

Please send an email to lxu@trustlook.com if you have any comments. Thanks.