May 24, 2021

VirusTotal APK Malware Detection Data - Week 21: 202100517-20210523

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210517_20210523.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.71% 0.01% 11843 3 35646 35
Avira 99.55% 0.00% 11825 0 35649 53
Avast-Mobile 99.51% 0.29% 11820 104 35545 58
Trustlook 99.36% 0.06% 11802 23 35626 76
AhnLab-V3 98.94% 0.03% 11752 11 35638 126
Fortinet 98.68% 0.01% 11721 5 35644 157
Kaspersky 98.45% 0.00% 11694 0 35649 184
Ikarus 98.43% 0.05% 11692 18 35631 186
CAT-QuickHeal 97.68% 0.01% 11602 3 35646 276
Sophos 93.64% 0.00% 11123 0 35649 755
K7GW 92.74% 0.06% 11016 20 35629 862
DrWeb 89.03% 0.10% 10575 34 35615 1303
Microsoft 82.84% 0.03% 9840 9 35640 2038
McAfee 73.94% 0.01% 8783 2 35647 3095
AVG 71.14% 0.01% 8450 2 35647 3428
Avast 70.71% 0.01% 8399 2 35647 3479
McAfee-GW-Edition 69.73% 0.00% 8282 1 35648 3596
Cyren 66.79% 0.01% 7933 2 35647 3945
ZoneAlarm 66.46% 0.00% 7894 1 35648 3984
AegisLab 63.66% 0.08% 7561 30 35619 4317
SymantecMobileInsight 55.75% 2.00% 6622 714 34935 5256
NANO-Antivirus 52.15% 0.05% 6194 17 35632 5684
Tencent 45.33% 0.05% 5384 19 35630 6494
Symantec 39.80% 0.01% 4728 3 35646 7150
ClamAV 37.63% 0.13% 4470 48 35601 7408
Alibaba 31.29% 0.01% 3717 4 35645 8161
Kingsoft 23.72% 0.00% 2818 1 35648 9060
MAX 22.49% 0.00% 2671 0 35649 9207
Zillya 18.02% 0.12% 2140 43 35606 9738
Comodo 17.96% 0.09% 2133 31 35618 9745
Rising 13.09% 0.04% 1555 15 35634 10323
GData 11.69% 0.00% 1389 0 35649 10489
TrendMicro-HouseCall 10.74% 0.02% 1276 7 35642 10602
TrendMicro 10.62% 0.04% 1262 13 35636 10616
Jiangmin 9.35% 0.61% 1110 219 35430 10768
F-Secure 6.92% 0.00% 822 0 35649 11056
Zoner 5.94% 0.04% 706 14 35635 11172
BitDefender 5.80% 0.00% 689 0 35649 11189
Emsisoft 5.73% 0.00% 681 0 35649 11197
VBA32 4.98% 0.01% 592 2 35647 11286
Yandex 4.58% 0.00% 544 0 35649 11334
Arcabit 3.12% 0.00% 370 0 35649 11508
MicroWorld-eScan 2.59% 0.00% 308 0 35649 11570
VIPRE 2.31% 0.00% 274 0 35649 11604
Panda 1.70% 0.00% 202 0 35649 11676
Antiy-AVL 0.75% 0.01% 89 2 35647 11789
Babable 0.57% 0.04% 68 13 35636 11810
Ad-Aware 0.41% 0.00% 49 0 35649 11829
F-Prot 0.21% 0.00% 25 0 35649 11853
Baidu 0.13% 0.02% 16 6 35643 11862
ViRobot 0.06% 0.00% 7 0 35649 11871
SentinelOne 0.04% 0.00% 5 0 35649 11873
SUPERAntiSpyware 0.03% 0.00% 4 0 35649 11874
Bkav 0.00% 0.00% 0 0 35649 11878
TotalDefense 0.00% 0.00% 0 0 35649 11878
nProtect 0.00% 0.00% 0 0 35649 11878
CMC 0.00% 0.00% 0 0 35649 11878
ALYac 0.00% 0.00% 0 0 35649 11878
Malwarebytes 0.00% 0.00% 0 0 35649 11878
CrowdStrike 0.00% 0.00% 0 0 35649 11878
K7AntiVirus 0.00% 0.00% 0 0 35649 11878
TheHacker 0.00% 0.00% 0 0 35649 11878
eScan 0.00% 0.00% 0 0 35649 11878
Invincea 0.00% 0.00% 0 0 35649 11878
Endgame 0.00% 0.00% 0 0 35649 11878
Webroot 0.00% 0.00% 0 0 35649 11878
AVware 0.00% 0.00% 0 0 35649 11878
Qihoo-360 0.00% 0.00% 0 0 35649 11878
TotalGoodware 35649
TotalMalware 11878
TotalSample 47527

Please send an email to lxu@trustlook.com if you have any comments. Thanks.