March 1, 2021

VirusTotal APK Malware Detection Data - Week 9: 20210222-20210228

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210222_20210228.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.66% 0.05% 20236 31 58996 69
ESET-NOD32 99.18% 0.02% 20139 11 59016 166
Trustlook 98.83% 0.16% 20067 96 58931 238
AhnLab-V3 98.08% 0.03% 19916 15 59012 389
ZoneAlarm 98.03% 0.01% 19905 3 59024 400
Avira 97.42% 0.00% 19782 0 59027 523
Fortinet 97.42% 0.01% 19781 6 59021 524
Avast-Mobile 97.08% 0.59% 19712 349 58678 593
McAfee 96.73% 0.00% 19642 0 59027 663
Kaspersky 95.93% 0.00% 19478 1 59026 827
Ikarus 95.67% 0.11% 19425 67 58960 880
McAfee-GW-Edition 95.13% 0.01% 19316 6 59021 989
AegisLab 93.19% 0.17% 18922 98 58929 1383
F-Secure 91.94% 0.00% 18668 0 59027 1637
DrWeb 89.86% 0.12% 18247 72 58955 2058
Microsoft 89.14% 0.03% 18099 16 59011 2206
Sophos 87.58% 0.02% 17784 14 59013 2521
CAT-QuickHeal 87.56% 0.01% 17779 5 59022 2526
NANO-Antivirus 84.56% 0.01% 17169 8 59019 3136
Qihoo-360 83.77% 0.03% 17009 16 59011 3296
SymantecMobileInsight 77.72% 2.17% 15781 1283 57744 4524
Symantec 75.71% 0.01% 15373 6 59021 4932
Antiy-AVL 73.99% 0.19% 15023 112 58915 5282
Cyren 71.54% 0.01% 14526 6 59021 5779
Alibaba 71.14% 0.00% 14445 2 59025 5860
Tencent 69.18% 0.06% 14048 37 58990 6257
Avast 68.28% 0.04% 13865 26 59001 6440
AVG 68.28% 0.04% 13865 26 59001 6440
MAX 65.20% 0.00% 13238 0 59027 7067
Comodo 49.82% 0.06% 10115 35 58992 10190
Zillya 45.79% 0.05% 9298 31 58996 11007
ClamAV 25.39% 0.19% 5156 111 58916 15149
Jiangmin 22.41% 0.29% 4550 173 58854 15755
GData 18.98% 0.00% 3853 0 59027 16452
Rising 17.42% 0.01% 3538 5 59022 16767
BitDefender 14.69% 0.00% 2983 0 59027 17322
Arcabit 14.59% 0.00% 2962 0 59027 17343
Emsisoft 14.51% 0.00% 2947 0 59027 17358
Yandex 13.14% 0.01% 2668 3 59024 17637
TrendMicro 12.99% 0.05% 2638 31 58996 17667
TrendMicro-HouseCall 12.91% 0.03% 2621 17 59010 17684
Kingsoft 12.13% 0.01% 2464 7 59020 17841
MicroWorld-eScan 10.27% 0.00% 2086 0 59027 18219
VBA32 7.75% 0.01% 1574 6 59021 18731
Ad-Aware 6.74% 0.00% 1369 0 59027 18936
Zoner 5.59% 0.07% 1135 39 58988 19170
VIPRE 4.72% 0.00% 959 0 59027 19346
Panda 2.01% 0.00% 408 0 59027 19897
TotalDefense 1.05% 0.00% 213 0 59027 20092
K7AntiVirus 0.21% 0.00% 43 0 59027 20262
Baidu 0.09% 0.01% 18 5 59022 20287
ViRobot 0.08% 0.00% 17 0 59027 20288
ALYac 0.06% 0.00% 13 1 59026 20292
Malwarebytes 0.05% 0.00% 10 0 59027 20295
SentinelOne 0.04% 0.00% 8 1 59026 20297
SUPERAntiSpyware 0.00% 0.00% 1 0 59027 20304
Bkav 0.00% 0.00% 0 0 59027 20305
nProtect 0.00% 0.00% 0 0 59027 20305
CMC 0.00% 0.00% 0 0 59027 20305
CrowdStrike 0.00% 0.00% 0 0 59027 20305
TheHacker 0.00% 0.00% 0 0 59027 20305
eScan 0.00% 0.00% 0 0 59027 20305
Babable 0.00% 0.00% 0 0 59027 20305
Invincea 0.00% 0.00% 0 0 59027 20305
F-Prot 0.00% 0.00% 0 0 59027 20305
Endgame 0.00% 0.00% 0 0 59027 20305
Webroot 0.00% 0.00% 0 0 59027 20305
AVware 0.00% 0.00% 0 0 59027 20305
TotalGoodware 59027
TotalMalware 20305
TotalSample 79332

Please send an email to lxu@trustlook.com if you have any comments. Thanks.