August 10, 2020

VirusTotal APK Malware Detection Data - Week 32: 20200803-20200809

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200803_20200809.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.72% 0.01% 40351 7 50627 113
K7GW 99.64% 0.16% 40317 80 50554 147
Trustlook 99.50% 0.18% 40261 92 50542 203
Avast-Mobile 98.91% 0.19% 40021 98 50536 443
McAfee 98.51% 0.01% 39860 3 50631 604
Avira 98.50% 0.00% 39856 0 50634 608
DrWeb 98.28% 0.19% 39768 95 50539 696
Fortinet 97.95% 0.03% 39635 13 50621 829
AhnLab-V3 96.77% 0.03% 39157 16 50618 1307
F-Secure 96.01% 0.00% 38848 0 50634 1616
ZoneAlarm 95.31% 0.02% 38565 10 50624 1899
Kaspersky 94.67% 0.00% 38307 1 50633 2157
Ikarus 93.57% 0.15% 37864 75 50559 2600
NANO-Antivirus 92.29% 0.03% 37344 17 50617 3120
Sophos 92.09% 0.03% 37262 15 50619 3202
Qihoo-360 90.12% 0.02% 36467 12 50622 3997
Symantec 87.19% 0.03% 35279 15 50619 5185
AVG 83.46% 0.05% 33770 27 50607 6694
CAT-QuickHeal 83.40% 0.07% 33746 35 50599 6718
Ad-Aware 0.25% 0.00% 101 0 50634 40363
McAfee-GW-Edition 0.00% 0.00% 0 0 50634 40464
TotalGoodware 50634
TotalMalware 40464
TotalSample 91098

Please send an email to lxu@trustlook.com if you have any comments. Thanks.