VirusTotal APK Malware Detection Data - Week 38: 20200914-20200920

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200914_20200920.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.75% 0.07% 14000 36 50963 35
ESET-NOD32 99.74% 0.01% 13999 5 50994 36
Avira 99.19% 0.00% 13921 0 50999 114
DrWeb 99.08% 0.19% 13906 96 50903 129
Trustlook 99.05% 0.11% 13901 56 50943 134
Fortinet 98.89% 0.03% 13879 16 50983 156
CAT-QuickHeal 98.37% 0.02% 13806 12 50987 229
Avast-Mobile 97.90% 0.17% 13740 89 50910 295
AhnLab-V3 97.46% 0.05% 13679 27 50972 356
McAfee 97.29% 0.00% 13655 1 50998 380
ZoneAlarm 97.29% 0.01% 13655 5 50994 380
Kaspersky 96.32% 0.01% 13518 3 50996 517
Ikarus 95.15% 0.15% 13355 75 50924 680
F-Secure 89.43% 0.00% 12551 0 50999 1484
NANO-Antivirus 79.47% 0.05% 11153 27 50972 2882
Sophos 69.14% 0.03% 9704 15 50984 4331
Symantec 68.86% 0.00% 9665 2 50997 4370
Qihoo-360 59.19% 0.02% 8307 8 50991 5728
AVG 37.49% 0.02% 5262 11 50988 8773
McAfee-GW-Edition 5.94% 0.00% 833 0 50999 13202
Ad-Aware 0.67% 0.00% 94 0 50999 13941
TotalGoodware 50999
TotalMalware 14035
TotalSample 65034

Please send an email to lxu@trustlook.com if you have any comments. Thanks.