May 4, 2020

VirusTotal APK Malware Detection Data - Week 18: 20200427-20200503

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200427_20200503.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
Trustlook 99.43% 0.20% 47139 143 72412 270
K7GW 99.33% 0.18% 47092 134 72421 317
Avast-Mobile 99.30% 0.27% 47076 199 72356 333
AhnLab-V3 99.10% 0.05% 46983 36 72519 426
Avira 98.66% 0.00% 46776 0 72555 633
ZoneAlarm 98.56% 0.01% 46728 10 72545 681
McAfee 98.45% 0.02% 46676 18 72537 733
ESET-NOD32 97.85% 0.04% 46388 27 72528 1021
Kaspersky 97.14% 0.01% 46055 8 72547 1354
Ikarus 97.11% 0.17% 46039 121 72434 1370
Qihoo-360 95.91% 0.02% 45471 13 72542 1938
Sophos 95.59% 0.04% 45319 26 72529 2090
F-Secure 93.91% 0.01% 44521 6 72549 2888
NANO-Antivirus 93.45% 0.05% 44303 38 72517 3106
DrWeb 93.31% 0.27% 44238 196 72359 3171
AVG 86.69% 0.10% 41101 73 72482 6308
Fortinet 74.53% 0.01% 35333 7 72548 12076
McAfee-GW-Edition 74.16% 0.00% 35159 1 72554 12250
CAT-QuickHeal 65.55% 0.05% 31075 39 72516 16334
Symantec 51.72% 0.01% 24520 5 72550 22889
Ad-Aware 0.35% 0.00% 164 0 72555 47245
TotalGoodware 72555
TotalMalware 47409
TotalSample 119964

Please send an email to lxu@trustlook.com if you have any comments. Thanks.