May 19, 2021

VirusTotal APK Malware Detection Data - Week 20: 202100510-20210516

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20210510_20210516.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative

Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.24% 0.01% 6635 5 41984 51
Trustlook 98.82% 0.19% 6607 81 41908 79
K7GW 98.80% 0.10% 6606 40 41949 80
Ikarus 98.76% 0.08% 6603 35 41954 83
Fortinet 98.73% 0.01% 6601 5 41984 85
Avast-Mobile 98.01% 0.32% 6553 133 41856 133
CAT-QuickHeal 97.89% 0.01% 6545 3 41986 141
Avira 97.50% 0.00% 6519 1 41988 167
DrWeb 97.17% 0.18% 6497 76 41913 189
AhnLab-V3 96.07% 0.03% 6423 12 41977 263
McAfee 93.82% 0.00% 6273 1 41988 413
Kaspersky 92.81% 0.01% 6205 3 41986 481
McAfee-GW-Edition 88.24% 0.03% 5900 11 41978 786
AegisLab 84.31% 0.10% 5637 40 41949 1049
SymantecMobileInsight 83.68% 2.37% 5595 994 40995 1091
Microsoft 82.37% 0.02% 5507 10 41979 1179
NANO-Antivirus 81.83% 0.04% 5471 17 41972 1215
Symantec 77.76% 0.05% 5199 22 41967 1487
Sophos 72.58% 0.01% 4853 5 41984 1833
Alibaba 68.98% 0.01% 4612 4 41985 2074
Tencent 66.36% 0.08% 4437 33 41956 2249
ZoneAlarm 57.40% 0.01% 3838 4 41985 2848
MAX 53.31% 0.00% 3564 0 41989 3122
Cyren 38.77% 0.01% 2592 3 41986 4094
AVG 34.37% 0.01% 2298 6 41983 4388
Avast 34.18% 0.01% 2285 6 41983 4401
Zillya 30.38% 0.10% 2031 42 41947 4655
Comodo 26.07% 0.06% 1743 27 41962 4943
Jiangmin 23.23% 0.75% 1553 316 41673 5133
GData 18.47% 0.00% 1235 0 41989 5451
BitDefender 18.07% 0.00% 1208 0 41989 5478
Emsisoft 17.77% 0.00% 1188 0 41989 5498
ClamAV 14.06% 0.12% 940 50 41939 5746
TrendMicro-HouseCall 9.24% 0.02% 618 7 41982 6068
Arcabit 9.12% 0.00% 610 0 41989 6076
Rising 8.57% 0.00% 573 2 41987 6113
Yandex 8.38% 0.01% 560 4 41985 6126
F-Secure 6.57% 0.00% 439 0 41989 6247
MicroWorld-eScan 6.22% 0.00% 416 0 41989 6270
Zoner 5.76% 0.00% 385 1 41988 6301
Antiy-AVL 4.86% 0.03% 325 12 41977 6361
Kingsoft 4.77% 0.02% 319 8 41981 6367
TrendMicro 1.94% 0.05% 130 19 41970 6556
VBA32 1.06% 0.01% 71 3 41986 6615
Ad-Aware 0.87% 0.00% 58 0 41989 6628
VIPRE 0.40% 0.00% 27 0 41989 6659
Babable 0.30% 0.04% 20 17 41972 6666
Panda 0.27% 0.00% 18 0 41989 6668
F-Prot 0.16% 0.00% 11 0 41989 6675
Baidu 0.12% 0.02% 8 9 41980 6678
ViRobot 0.07% 0.00% 5 0 41989 6681
ALYac 0.03% 0.00% 2 0 41989 6684
Malwarebytes 0.01% 0.00% 1 0 41989 6685
SentinelOne 0.01% 0.00% 1 0 41989 6685
Bkav 0.00% 0.00% 0 1 41988 6686
TotalDefense 0.00% 0.00% 0 0 41989 6686
nProtect 0.00% 0.00% 0 0 41989 6686
CMC 0.00% 0.00% 0 0 41989 6686
CrowdStrike 0.00% 0.00% 0 0 41989 6686
K7AntiVirus 0.00% 0.00% 0 0 41989 6686
TheHacker 0.00% 0.00% 0 0 41989 6686
eScan 0.00% 0.00% 0 0 41989 6686
SUPERAntiSpyware 0.00% 0.00% 0 0 41989 6686
Invincea 0.00% 0.00% 0 0 41989 6686
Endgame 0.00% 0.00% 0 0 41989 6686
Webroot 0.00% 0.00% 0 0 41989 6686
AVware 0.00% 0.00% 0 0 41989 6686
Qihoo-360 0.00% 0.00% 0 0 41989 6686
TotalGoodware 41989
TotalMalware 6686
TotalSample 48675

Please send an email to lxu@trustlook.com if you have any comments. Thanks.