June 22, 2021

VirusTotal APK Malware Detection Data - Week 24: 202100607-20210613

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210607_20210613.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
Trustlook 99.74% 0.11% 11381 32 27836 30
Avira 99.71% 0.00% 11378 0 27868 33
ESET-NOD32 99.60% 0.01% 11365 2 27866 46
K7GW 99.54% 0.10% 11358 29 27839 53
Fortinet 99.50% 0.00% 11354 1 27867 57
AhnLab-V3 99.47% 0.04% 11350 11 27857 61
Avast-Mobile 99.33% 0.27% 11335 76 27792 76
DrWeb 99.19% 0.12% 11319 34 27834 92
Ikarus 98.98% 0.06% 11295 17 27851 116
CAT-QuickHeal 98.05% 0.01% 11189 2 27866 222
Kaspersky 98.05% 0.00% 11188 1 27867 223
Sophos 91.68% 0.01% 10462 4 27864 949
McAfee 88.27% 0.00% 10072 1 27867 1339
McAfee-GW-Edition 82.93% 0.01% 9463 3 27865 1948
AegisLab 78.36% 0.13% 8942 37 27831 2469
Microsoft 73.68% 0.02% 8408 6 27862 3003
ZoneAlarm 72.20% 0.00% 8239 0 27868 3172
Cyren 59.59% 0.00% 6800 1 27867 4611
SymantecMobileInsight 49.57% 2.26% 5657 631 27237 5754
Symantec 49.25% 0.00% 5620 0 27868 5791
NANO-Antivirus 47.94% 0.05% 5471 13 27855 5940
Alibaba 46.74% 0.01% 5334 2 27866 6077
Tencent 45.97% 0.04% 5246 11 27857 6165
MAX 35.84% 0.00% 4090 0 27868 7321
Zillya 32.42% 0.09% 3700 26 27842 7711
AVG 30.16% 0.01% 3441 2 27866 7970
Avast 29.90% 0.01% 3412 2 27866 7999
Kingsoft 25.34% 0.00% 2891 1 27867 8520
Antiy-AVL 19.25% 0.08% 2197 21 27847 9214
ClamAV 16.38% 0.06% 1869 16 27852 9542
Zoner 8.69% 0.01% 992 4 27864 10419
F-Secure 6.72% 0.00% 767 0 27868 10644
Comodo 6.50% 0.07% 742 19 27849 10669
Yandex 4.93% 0.00% 562 1 27867 10849
Jiangmin 3.67% 0.80% 419 224 27644 10992
GData 2.30% 0.00% 263 0 27868 11148
BitDefender 2.10% 0.00% 240 0 27868 11171
Emsisoft 2.08% 0.00% 237 0 27868 11174
MicroWorld-eScan 1.29% 0.00% 147 0 27868 11264
Rising 1.22% 0.02% 139 6 27862 11272
Arcabit 1.16% 0.00% 132 1 27867 11279
TrendMicro 1.03% 0.01% 118 2 27866 11293
TrendMicro-HouseCall 0.98% 0.00% 112 0 27868 11299
Ad-Aware 0.55% 0.00% 63 0 27868 11348
VBA32 0.49% 0.01% 56 3 27865 11355
Babable 0.15% 0.00% 17 1 27867 11394
Panda 0.06% 0.00% 7 0 27868 11404
Baidu 0.04% 0.01% 5 3 27865 11406
SentinelOne 0.04% 0.00% 5 0 27868 11406
ViRobot 0.03% 0.00% 3 1 27867 11408
F-Prot 0.03% 0.00% 3 0 27868 11408
VIPRE 0.02% 0.00% 2 0 27868 11409
K7AntiVirus 0.01% 0.00% 1 0 27868 11410
Bkav 0.00% 0.00% 0 0 27868 11411
TotalDefense 0.00% 0.00% 0 0 27868 11411
nProtect 0.00% 0.00% 0 0 27868 11411
CMC 0.00% 0.00% 0 0 27868 11411
ALYac 0.00% 0.00% 0 1 27867 11411
Malwarebytes 0.00% 0.00% 0 0 27868 11411
CrowdStrike 0.00% 0.00% 0 0 27868 11411
TheHacker 0.00% 0.00% 0 0 27868 11411
eScan 0.00% 0.00% 0 0 27868 11411
SUPERAntiSpyware 0.00% 0.00% 0 0 27868 11411
Invincea 0.00% 0.00% 0 0 27868 11411
Endgame 0.00% 0.00% 0 0 27868 11411
Webroot 0.00% 0.00% 0 0 27868 11411
AVware 0.00% 0.00% 0 0 27868 11411
Qihoo-360 0.00% 0.00% 0 0 27868 11411
TotalGoodware 27868
TotalMalware 11411
TotalSample 39279

Please send an email to lxu@trustlook.com if you have any comments. Thanks.