Enhancing DNS-over-HTTPS Traffic Classification in Heterogeneous Networks Through Latent Space Analysis with a Tabular-Variational Autoencoder and Self-Attention Classifier Model

Ravi Veerabhadrappa; Poornima Athikatte Sampigerayappa

doi:10.47852/bonviewAIA52025552

Authors

Ravi Veerabhadrappa Department of Computer Science and Engineering, Siddaganga Institute of Technology, India https://orcid.org/0000-0003-2238-4642
Poornima Athikatte Sampigerayappa Department of Computer Science and Engineering, Siddaganga Institute of Technology, India https://orcid.org/0000-0001-9458-4251

DOI:

https://doi.org/10.47852/bonviewAIA52025552

Keywords:

DNS over HTTPS, Tab-VAE, self-attention classifier, heterogeneous networks

Abstract

Cybersecurity threats and attacks are increasing day by day, bringing real focus on Domain Name System (DNS)–based data exfiltration—a stealth technique used by attackers to steal sensitive information from compromised networks. DNS query exchange is the initial part of any data exchange in the Internet and is the most neglected in traditional monitoring systems. These enable attackers to create covert channels to carry out various advanced persistent threats and unauthorized exfiltration attempts. In this research study, we present a novel detection approach of these DNS patterns through low-dimensional latent representations extracted via a Tabular-Variational AutoEncoder (Tab-VAE ), specifically tailored for DNS-over-HTTPS (DoH) traffic. The latent space obtained by the Tab-VAE is subsequently fed into a multi-head self-attention classifier to perform a multi-class classification. We evaluated our experiments using the BCCC-CIC-Bell-DNS-2024 dataset, which provides a realistic snapshot of DoH traffic patterns. Notably, the proposed model demonstrated robust generalization across varying batch sizes and achieved competitive performance metrics with an improved accuracy of 80% and precision score of 75% for a batch size of 128. These findings highlight the potential of advanced machine learning architectures in reinforcing cybersecurity posture. By integrating such techniques, organizations can improve the detection of covert DNS-based attacks and better protect sensitive assets against emerging threats.

Received: 28 February 2025 | Revised: 4 July 2025 | Accepted: 22 July 2025

Conflicts of Interest

The authors declare that they have no conflicts of interest in this work.

Data Availability Statement

The data supporting the findings of this study are openly available in Behaviour-Centric Cybersecurity Center (BCCC) at https://www.yorku.ca/research/bccc/ucs-technical/cybersecurity-datasets-cds/.

Author Contribution Statement

Ravi Veerabhadrappa: Conceptualization, Methodology, Software, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Visualization, Project administration. Poornima Athikatte Sampigerayappa: Validation, Writing – review and editing, Supervision, Project administration.