October 26, 2020

VirusTotal APK Malware Detection Data - Week 43: 20201019-20201025

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201019_20201025.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.54% 0.01% 13531 9 67922 63
K7GW 99.04% 0.14% 13463 97 67834 131
Avira 98.43% 0.00% 13380 2 67929 214
Trustlook 98.00% 0.14% 13322 93 67838 272
ZoneAlarm 97.85% 0.00% 13302 2 67929 292
CAT-QuickHeal 97.47% 0.02% 13250 14 67917 344
Fortinet 97.33% 0.02% 13231 11 67920 363
Avast-Mobile 96.95% 0.17% 13179 114 67817 415
Kaspersky 96.65% 0.00% 13139 0 67931 455
DrWeb 96.44% 0.27% 13110 184 67747 484
AhnLab-V3 94.30% 0.04% 12819 26 67905 775
Ikarus 93.79% 0.17% 12750 118 67813 844
McAfee 90.58% 0.00% 12314 1 67930 1280
McAfee-GW-Edition 87.66% 0.02% 11917 14 67917 1677
NANO-Antivirus 87.58% 0.06% 11905 41 67890 1689
Qihoo-360 82.59% 0.02% 11227 13 67918 2367
Sophos 81.80% 0.03% 11120 22 67909 2474
Symantec 74.29% 0.01% 10099 6 67925 3495
AVG 58.95% 0.03% 8013 20 67911 5581
F-Secure 51.74% 0.00% 7033 0 67931 6561
Ad-Aware 1.41% 0.00% 192 0 67931 13402
TotalGoodware 67931
TotalMalware 13594
TotalSample 81525

Please send an email to lxu@trustlook.com if you have any comments. Thanks.