VirusTotal APK Malware Detection Data - Week 25: 202100614-20210620

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210614_20210620.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
K7GW 99.78% 0.12% 10383 33 27602 23
Avira 99.72% 0.00% 10377 1 27634 29
ESET-NOD32 99.70% 0.01% 10375 4 27631 31
AhnLab-V3 99.60% 0.03% 10364 9 27626 42
Fortinet 99.60% 0.01% 10364 3 27632 42
Trustlook 99.55% 0.19% 10359 52 27583 47
Avast-Mobile 99.02% 0.25% 10304 69 27566 102
Ikarus 99.02% 0.06% 10304 17 27618 102
CAT-QuickHeal 98.58% 0.01% 10258 3 27632 148
Kaspersky 97.75% 0.00% 10172 0 27635 234
Sophos 91.35% 0.00% 9506 0 27635 900
McAfee 90.64% 0.00% 9432 1 27634 974
DrWeb 89.68% 0.25% 9332 69 27566 1074
McAfee-GW-Edition 85.08% 0.00% 8853 1 27634 1553
AegisLab 82.19% 0.13% 8553 37 27598 1853
Microsoft 80.55% 0.01% 8382 4 27631 2024
Cyren 72.27% 0.01% 7520 4 27631 2886
ZoneAlarm 71.33% 0.01% 7423 2 27633 2983
NANO-Antivirus 69.60% 0.05% 7243 14 27621 3163
Antiy-AVL 53.33% 0.10% 5550 28 27607 4856
AVG 46.67% 0.01% 4857 2 27633 5549
Avast 46.29% 0.01% 4817 2 27633 5589
Zillya 46.22% 0.10% 4810 29 27606 5596
SymantecMobileInsight 46.10% 2.62% 4797 723 26912 5609
Symantec 40.61% 0.01% 4226 4 27631 6180
Alibaba 35.44% 0.01% 3688 2 27633 6718
Kingsoft 28.58% 0.01% 2974 2 27633 7432
MAX 20.70% 0.00% 2154 0 27635 8252
Tencent 19.88% 0.03% 2069 9 27626 8337
ClamAV 19.01% 0.06% 1978 16 27619 8428
Jiangmin 8.50% 0.86% 884 239 27396 9522
Yandex 8.03% 0.00% 836 1 27634 9570
Comodo 7.82% 0.08% 814 23 27612 9592
Zoner 7.74% 0.01% 805 3 27632 9601
F-Secure 7.22% 0.00% 751 0 27635 9655
GData 2.17% 0.00% 226 0 27635 10180
BitDefender 1.96% 0.00% 204 0 27635 10202
Emsisoft 1.90% 0.00% 198 0 27635 10208
TrendMicro-HouseCall 1.81% 0.00% 188 0 27635 10218
MicroWorld-eScan 1.15% 0.00% 120 0 27635 10286
Rising 1.12% 0.01% 117 3 27632 10289
Arcabit 1.01% 0.00% 105 0 27635 10301
TrendMicro 0.84% 0.01% 87 3 27632 10319
Ad-Aware 0.52% 0.00% 54 0 27635 10352
Babable 0.23% 0.01% 24 2 27633 10382
VBA32 0.16% 0.00% 17 1 27634 10389
Panda 0.07% 0.00% 7 0 27635 10399
K7AntiVirus 0.03% 0.00% 3 0 27635 10403
VIPRE 0.02% 0.00% 2 0 27635 10404
SentinelOne 0.02% 0.00% 2 0 27635 10404
ALYac 0.01% 0.00% 1 0 27635 10405
Baidu 0.01% 0.01% 1 3 27632 10405
ViRobot 0.01% 0.00% 1 0 27635 10405
Bkav 0.00% 0.00% 0 0 27635 10406
TotalDefense 0.00% 0.00% 0 0 27635 10406
nProtect 0.00% 0.00% 0 0 27635 10406
CMC 0.00% 0.00% 0 0 27635 10406
Malwarebytes 0.00% 0.00% 0 0 27635 10406
CrowdStrike 0.00% 0.00% 0 0 27635 10406
TheHacker 0.00% 0.00% 0 0 27635 10406
eScan 0.00% 0.00% 0 0 27635 10406
SUPERAntiSpyware 0.00% 0.00% 0 0 27635 10406
Invincea 0.00% 0.00% 0 0 27635 10406
F-Prot 0.00% 0.00% 0 0 27635 10406
Endgame 0.00% 0.00% 0 0 27635 10406
Webroot 0.00% 0.00% 0 0 27635 10406
AVware 0.00% 0.00% 0 0 27635 10406
Qihoo-360 0.00% 0.00% 0 0 27635 10406
TotalGoodware 27635
TotalMalware 10406
TotalSample 38041

Please send an email to lxu@trustlook.com if you have any comments. Thanks.