July 20, 2020

VirusTotal APK Malware Detection Data - Week 29: 20200713-20200719

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200713_20200719.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.81% 0.02% 14941 9 57414 29
K7GW 99.64% 0.05% 14916 27 57396 54
Fortinet 99.56% 0.03% 14904 15 57408 66
Trustlook 98.74% 0.25% 14782 144 57279 188
Avast-Mobile 98.41% 0.13% 14732 73 57350 238
ZoneAlarm 98.19% 0.01% 14699 3 57420 271
DrWeb 97.77% 0.05% 14636 29 57394 334
AhnLab-V3 97.63% 0.05% 14615 28 57395 355
Avira 97.58% 0.00% 14607 0 57423 363
Kaspersky 97.29% 0.00% 14564 2 57421 406
F-Secure 94.40% 0.00% 14132 0 57423 838
Ikarus 94.34% 0.06% 14123 34 57389 847
McAfee 88.93% 0.00% 13313 2 57421 1657
CAT-QuickHeal 88.52% 0.01% 13251 4 57419 1719
Sophos 87.17% 0.02% 13050 9 57414 1920
Qihoo-360 87.17% 0.01% 13049 8 57415 1921
NANO-Antivirus 79.78% 0.01% 11943 8 57415 3027
Symantec 77.73% 0.01% 11636 5 57418 3334
AVG 72.27% 0.07% 10819 39 57384 4151
Ad-Aware 0.38% 0.00% 57 0 57423 14913
McAfee-GW-Edition 0.00% 0.00% 0 0 57423 14970
TotalGoodware 57423
TotalMalware 14970
TotalSample 72393

Please send an email to lxu@trustlook.com if you have any comments. Thanks.