February 23, 2021

VirusTotal APK Malware Detection Data - Week 8: 20210215-20210221

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210215_20210221.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.65% 0.08% 8603 41 48861 30
ESET-NOD32 99.63% 0.02% 8601 9 48893 32
Avast-Mobile 98.83% 0.51% 8532 251 48651 101
Fortinet 98.53% 0.01% 8506 6 48896 127
Trustlook 98.45% 0.07% 8499 33 48869 134
Avira 98.39% 0.00% 8494 1 48901 139
DrWeb 98.18% 0.13% 8476 64 48838 157
AhnLab-V3 97.89% 0.03% 8451 16 48886 182
ZoneAlarm 97.21% 0.01% 8392 5 48897 241
Ikarus 96.54% 0.12% 8334 60 48842 299
Kaspersky 95.12% 0.00% 8212 1 48901 421
F-Secure 94.37% 0.00% 8147 2 48900 486
McAfee 93.73% 0.01% 8092 3 48899 541
McAfee-GW-Edition 90.51% 0.01% 7814 3 48899 819
AegisLab 89.63% 0.12% 7738 60 48842 895
Sophos 87.66% 0.01% 7568 7 48895 1065
Microsoft 84.43% 0.01% 7289 6 48896 1344
Antiy-AVL 81.25% 0.22% 7014 109 48793 1619
NANO-Antivirus 77.18% 0.04% 6663 21 48881 1970
Cyren 64.99% 0.00% 5611 1 48901 3022
CAT-QuickHeal 55.59% 0.01% 4799 5 48897 3834
Qihoo-360 54.52% 0.01% 4707 4 48898 3926
Zillya 47.27% 0.10% 4081 49 48853 4552
SymantecMobileInsight 38.67% 2.28% 3338 1114 47788 5295
Symantec 37.59% 0.01% 3245 3 48899 5388
Comodo 30.55% 0.09% 2637 43 48859 5996
Avast 30.42% 0.03% 2626 15 48887 6007
AVG 30.42% 0.03% 2626 15 48887 6007
Alibaba 30.41% 0.01% 2625 6 48896 6008
Tencent 29.82% 0.05% 2574 26 48876 6059
MAX 26.13% 0.00% 2256 0 48902 6377
Yandex 23.14% 0.00% 1998 2 48900 6635
ClamAV 15.45% 0.19% 1334 94 48808 7299
Jiangmin 11.60% 0.32% 1001 158 48744 7632
Kingsoft 9.21% 0.00% 795 1 48901 7838
GData 6.67% 0.00% 576 0 48902 8057
BitDefender 5.88% 0.00% 508 0 48902 8125
Arcabit 5.77% 0.00% 498 0 48902 8135
Emsisoft 5.70% 0.00% 492 0 48902 8141
Rising 4.52% 0.01% 390 4 48898 8243
Zoner 3.75% 0.00% 324 2 48900 8309
TrendMicro 3.36% 0.05% 290 26 48876 8343
MicroWorld-eScan 3.31% 0.00% 286 0 48902 8347
TrendMicro-HouseCall 2.98% 0.04% 257 22 48880 8376
VBA32 2.06% 0.02% 178 10 48892 8455
Ad-Aware 1.15% 0.00% 99 0 48902 8534
TotalDefense 0.76% 0.00% 66 0 48902 8567
VIPRE 0.56% 0.00% 48 0 48902 8585
Panda 0.53% 0.00% 46 0 48902 8587
Baidu 0.29% 0.02% 25 10 48892 8608
SentinelOne 0.21% 0.00% 18 0 48902 8615
K7AntiVirus 0.19% 0.00% 16 0 48902 8617
ViRobot 0.19% 0.00% 16 1 48901 8617
ALYac 0.13% 0.00% 11 1 48901 8622
SUPERAntiSpyware 0.06% 0.00% 5 0 48902 8628
CMC 0.05% 0.00% 4 0 48902 8629
Malwarebytes 0.05% 0.00% 4 0 48902 8629
Bkav 0.00% 0.00% 0 2 48900 8633
nProtect 0.00% 0.00% 0 0 48902 8633
CrowdStrike 0.00% 0.00% 0 0 48902 8633
TheHacker 0.00% 0.00% 0 0 48902 8633
eScan 0.00% 0.00% 0 0 48902 8633
Babable 0.00% 0.00% 0 0 48902 8633
Invincea 0.00% 0.00% 0 0 48902 8633
F-Prot 0.00% 0.00% 0 0 48902 8633
Endgame 0.00% 0.00% 0 0 48902 8633
Webroot 0.00% 0.00% 0 0 48902 8633
AVware 0.00% 0.00% 0 0 48902 8633
TotalGoodware 48902
TotalMalware 8633
TotalSample 57535

Please send an email to lxu@trustlook.com if you have any comments. Thanks.