September 8, 2020

VirusTotal APK Malware Detection Data - Week 36: 20200831-20200906

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200831_20200906.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.63% 0.06% 9453 31 54202 35
K7GW 99.50% 0.13% 9441 68 54165 47
CAT-QuickHeal 98.82% 0.01% 9376 6 54227 112
Trustlook 98.78% 0.15% 9372 83 54150 116
Avira 98.76% 0.00% 9370 0 54233 118
Fortinet 98.44% 0.01% 9340 8 54225 148
DrWeb 98.21% 0.27% 9318 148 54085 170
Avast-Mobile 97.21% 0.21% 9223 115 54118 265
AhnLab-V3 96.75% 0.05% 9180 27 54206 308
McAfee 96.33% 0.01% 9140 5 54228 348
Ikarus 95.00% 0.18% 9014 97 54136 474
ZoneAlarm 94.03% 0.01% 8922 3 54230 566
Kaspersky 93.55% 0.00% 8876 2 54231 612
F-Secure 92.86% 0.00% 8811 2 54231 677
NANO-Antivirus 83.60% 0.06% 7932 32 54201 1556
Sophos 66.23% 0.04% 6284 19 54214 3204
Symantec 58.03% 0.02% 5506 11 54222 3982
Qihoo-360 55.80% 0.03% 5294 16 54217 4194
AVG 48.87% 0.07% 4637 40 54193 4851
Ad-Aware 0.93% 0.00% 88 0 54233 9400
McAfee-GW-Edition 0.00% 0.00% 0 0 54233 9488
TotalGoodware 54233
TotalMalware 9488
TotalSample 63721

Please send an email to lxu@trustlook.com if you have any comments. Thanks.