VirusTotal APK Malware Detection Data - Week 23: 202100531-20210606

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210531_20210606.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.69% 0.01% 7472 4 29425 23
Trustlook 99.61% 0.21% 7466 63 29366 29
Avira 99.61% 0.00% 7466 0 29429 29
Avast-Mobile 99.27% 0.51% 7440 150 29279 55
DrWeb 99.11% 0.14% 7428 42 29387 67
AhnLab-V3 99.11% 0.04% 7428 11 29418 67
Ikarus 99.09% 0.07% 7427 22 29407 68
Fortinet 98.79% 0.01% 7404 3 29426 91
Kaspersky 98.08% 0.00% 7351 1 29428 144
K7GW 97.91% 0.10% 7338 30 29399 157
CAT-QuickHeal 97.79% 0.00% 7329 1 29428 166
Sophos 93.62% 0.02% 7017 5 29424 478
McAfee 89.17% 0.00% 6683 1 29428 812
McAfee-GW-Edition 82.66% 0.01% 6195 2 29427 1300
Microsoft 78.44% 0.02% 5879 7 29422 1616
AegisLab 73.54% 0.14% 5512 42 29387 1983
ZoneAlarm 70.34% 0.00% 5272 0 29429 2223
Cyren 47.44% 0.00% 3556 0 29429 3939
NANO-Antivirus 46.88% 0.04% 3514 12 29417 3981
SymantecMobileInsight 44.64% 2.28% 3346 671 28758 4149
Tencent 39.61% 0.05% 2969 14 29415 4526
Symantec 37.38% 0.01% 2802 2 29427 4693
AVG 36.70% 0.01% 2751 4 29425 4744
Avast 36.41% 0.01% 2729 4 29425 4766
Alibaba 34.84% 0.00% 2611 1 29428 4884
Zillya 34.68% 0.10% 2599 30 29399 4896
Kingsoft 29.30% 0.00% 2196 1 29428 5299
MAX 28.01% 0.00% 2099 1 29428 5396
ClamAV 17.67% 0.05% 1324 15 29414 6171
Comodo 10.25% 0.03% 768 8 29421 6727
F-Secure 8.90% 0.00% 667 0 29429 6828
Zoner 5.00% 0.03% 375 8 29421 7120
Jiangmin 4.86% 0.78% 364 229 29200 7131
Yandex 4.32% 0.01% 324 2 29427 7171
Rising 3.83% 0.02% 287 5 29424 7208
GData 3.76% 0.00% 282 0 29429 7213
BitDefender 3.16% 0.00% 237 0 29429 7258
Emsisoft 3.11% 0.00% 233 0 29429 7262
Arcabit 2.12% 0.00% 159 0 29429 7336
TrendMicro-HouseCall 2.11% 0.00% 158 1 29428 7337
MicroWorld-eScan 2.09% 0.00% 157 0 29429 7338
TrendMicro 1.87% 0.01% 140 4 29425 7355
Antiy-AVL 0.92% 0.00% 69 1 29428 7426
Ad-Aware 0.75% 0.00% 56 0 29429 7439
VBA32 0.73% 0.00% 55 1 29428 7440
Panda 0.35% 0.00% 26 0 29429 7469
Babable 0.32% 0.01% 24 2 29427 7471
VIPRE 0.12% 0.00% 9 0 29429 7486
Baidu 0.11% 0.03% 8 8 29421 7487
ViRobot 0.09% 0.00% 7 0 29429 7488
SentinelOne 0.07% 0.00% 5 0 29429 7490
F-Prot 0.07% 0.00% 5 0 29429 7490
K7AntiVirus 0.03% 0.00% 2 0 29429 7493
Bkav 0.00% 0.00% 0 0 29429 7495
TotalDefense 0.00% 0.00% 0 0 29429 7495
nProtect 0.00% 0.00% 0 0 29429 7495
CMC 0.00% 0.00% 0 0 29429 7495
ALYac 0.00% 0.00% 0 0 29429 7495
Malwarebytes 0.00% 0.00% 0 0 29429 7495
CrowdStrike 0.00% 0.00% 0 0 29429 7495
TheHacker 0.00% 0.00% 0 0 29429 7495
eScan 0.00% 0.00% 0 0 29429 7495
SUPERAntiSpyware 0.00% 0.00% 0 0 29429 7495
Invincea 0.00% 0.00% 0 0 29429 7495
Endgame 0.00% 0.00% 0 0 29429 7495
Webroot 0.00% 0.00% 0 0 29429 7495
AVware 0.00% 0.00% 0 0 29429 7495
Qihoo-360 0.00% 0.00% 0 0 29429 7495
TotalGoodware 29429
TotalMalware 7495
TotalSample 36924

Please send an email to lxu@trustlook.com if you have any comments. Thanks.