July 5, 2021

VirusTotal APK Malware Detection Data - Week 27: 202100628-20210704

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210628_20210704.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
K7GW 99.80% 0.08% 8095 24 28416 16
ESET-NOD32 99.70% 0.02% 8087 5 28435 24
Avira 99.65% 0.00% 8083 0 28440 28
Fortinet 99.26% 0.00% 8051 1 28439 60
Trustlook 99.21% 0.07% 8047 19 28421 64
Ikarus 98.83% 0.05% 8016 13 28427 95
DrWeb 98.56% 0.07% 7994 21 28419 117
AhnLab-V3 98.03% 0.02% 7951 7 28433 160
Kaspersky 97.20% 0.00% 7884 1 28439 227
CAT-QuickHeal 96.20% 0.01% 7803 2 28438 308
Avast-Mobile 93.61% 0.45% 7593 128 28312 518
McAfee 91.01% 0.00% 7382 0 28440 729
Sophos 90.88% 0.01% 7371 2 28438 740
McAfee-GW-Edition 84.58% 0.00% 6860 0 28440 1251
AegisLab 80.45% 0.11% 6525 30 28410 1586
Microsoft 76.44% 0.01% 6200 4 28436 1911
Cyren 74.82% 0.02% 6069 7 28433 2042
ZoneAlarm 72.12% 0.00% 5850 0 28440 2261
NANO-Antivirus 71.19% 0.02% 5774 6 28434 2337
Symantec 54.84% 0.01% 4448 3 28437 3663
SymantecMobileInsight 52.10% 1.99% 4226 567 27873 3885
Alibaba 46.38% 0.00% 3762 1 28439 4349
Tencent 43.11% 0.03% 3497 9 28431 4614
Zillya 37.97% 0.09% 3080 27 28413 5031
Kingsoft 37.37% 0.00% 3031 1 28439 5080
Antiy-AVL 34.64% 0.13% 2810 38 28402 5301
MAX 31.48% 0.00% 2553 0 28440 5558
AVG 18.62% 0.02% 1510 5 28435 6601
Avast 18.54% 0.02% 1504 5 28435 6607
F-Secure 7.90% 0.00% 641 0 28440 7470
Comodo 7.41% 0.06% 601 16 28424 7510
Zoner 5.57% 0.00% 452 0 28440 7659
Jiangmin 4.71% 0.73% 382 209 28231 7729
ClamAV 4.62% 0.14% 375 40 28400 7736
Yandex 4.46% 0.01% 362 2 28438 7749
GData 3.18% 0.00% 258 0 28440 7853
BitDefender 3.09% 0.00% 251 0 28440 7860
Emsisoft 3.06% 0.00% 248 0 28440 7863
Qihoo-360 2.58% 0.07% 209 20 28420 7902
VBA32 1.98% 0.01% 161 2 28438 7950
MicroWorld-eScan 1.86% 0.00% 151 0 28440 7960
Rising 1.86% 0.01% 151 2 28438 7960
Arcabit 1.79% 0.00% 145 0 28440 7966
TrendMicro-HouseCall 1.71% 0.00% 139 0 28440 7972
TrendMicro 1.63% 0.00% 132 1 28439 7979
Ad-Aware 0.84% 0.00% 68 0 28440 8043
Panda 0.67% 0.00% 54 0 28440 8057
ViRobot 0.44% 0.00% 36 0 28440 8075
Baidu 0.41% 0.00% 33 1 28439 8078
Babable 0.31% 0.01% 25 4 28436 8086
SentinelOne 0.12% 0.00% 10 0 28440 8101
ALYac 0.05% 0.00% 4 0 28440 8107
Malwarebytes 0.04% 0.00% 3 0 28440 8108
SUPERAntiSpyware 0.01% 0.00% 1 0 28440 8110
VIPRE 0.01% 0.00% 1 0 28440 8110
F-Prot 0.01% 0.00% 1 0 28440 8110
Bkav 0.00% 0.00% 0 0 28440 8111
TotalDefense 0.00% 0.00% 0 0 28440 8111
nProtect 0.00% 0.00% 0 0 28440 8111
CMC 0.00% 0.00% 0 0 28440 8111
CrowdStrike 0.00% 0.00% 0 0 28440 8111
K7AntiVirus 0.00% 0.00% 0 0 28440 8111
TheHacker 0.00% 0.00% 0 0 28440 8111
eScan 0.00% 0.00% 0 0 28440 8111
Invincea 0.00% 0.00% 0 0 28440 8111
Endgame 0.00% 0.00% 0 0 28440 8111
Webroot 0.00% 0.00% 0 0 28440 8111
AVware 0.00% 0.00% 0 0 28440 8111
TotalGoodware 28440
TotalMalware 8111
TotalSample 36551

Please send an email to lxu@trustlook.com if you have any comments. Thanks.