Kleyton da Costa

When Should Graph Attention Be Sparse? Learning a Per-Edge Tsallis Index
Kleyton da Costa, Bernardo Modenesi.
[ArXiv 2026] [PDF]

Graph attention normalizes neighborhood scores with softmax, the maximum-entropy choice under Shannon statistics. But homophilic and heterophilic graphs want different attention shapes, and one fixed normalization cannot serve both. We propose LTGA (Learnable Tsallis Graph Attention), a graph attention layer whose Tsallis entropic index q is learned jointly with the weights, interpolating continuously between heavy-tailed (q<1), softmax (q=1) and compact-support (q>1) attention at four granularities from a global scalar to a per-edge index, under a bounded reparameterization that starts every model at the GAT baseline. Across eight benchmarks at ten seeds, LTGA-Edge takes the best average rank (2.75), but the omnibus test does not reject (p=0.199) and learning q does not beat searching it: a validation-tuned frozen grid reaches 61.4%, tuned α-entmax 62.2% and a capacity-matched q≡1 control 62.0%, against 61.7% for LTGA-Edge. What the learned index buys is one run instead of a grid, and an interpretable mechanism: where q leaves 1, it prunes 42% of attention coefficients to exactly zero, and those edges are selectively the wrong ones, restoring them costs 7.1 points, while random pruning at the same rate costs 13.0 more. Project page: https://kleyt0n.github.io/ltga

Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects
Seonglae Cho, Zekun Wu, Kleyton da Costa, Rishi Kalra, Ilham Wicaksono, Adriano Koshiyama.
[ArXiv 2026]

Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet whether a feature's causal role is stable across SAE families remains untested. Single-token features that activate on one vocabulary item provide the diagnostic case where ground truth permits direct comparison. We analyze 3.9M features across six models and three SAE families using zero-ablation at full layer depth. Single-token features cluster 4.7x tighter in decoder space and concentrate in early layers (Layer 0 in GPT2-Small; L0-L4 in Gemma). Ablating them yields Benjamini-Hochberg-significant logit reductions in 178 of 208 full-layer conditions, with depth controlling whether damage cascades downstream or shapes the output directly. Cross-family causal differences exceed within-family scale effects: on the same base model, GemmaScope and BatchTopK features remain causally anchored, while LlamaScope features are locally redundant. The target token's rank recovers to within 2x baseline 96-98% of the time after the same ablation, and a controlled activation-function comparison reverses sign within the same model, leaving training recipe as the residual candidate. Cross-family interpretability claims are therefore sensitive to training methodology, not just activation function or scale.

Divergence-Guided Particle Swarm Optimization
Kleyton da Costa, Bernardo Modenesi, Ivan F. M. Menezes, Hélio Lopes.
[ArXiv 2026]

Particle Swarm Optimization (PSO) is susceptible to premature convergence when the swarm collapses around the global best, particularly on multimodal landscapes in higher dimensions. We propose Divergence-guided PSO (DPSO), which augments the velocity update with a modulation term that repels particles whose personal bests have converged near the global best. The repulsion is gated by a Gaussian similarity kernel, which we prove is equivalent to an exponentially decaying function of the KL divergence between Gaussian-embedded personal and global bests, connecting the mechanism to the family of f-divergences and providing a principled basis for kernel design. Experiments on 36 benchmark functions (15 unimodal, 21 multimodal) across dimensions D ∈ 10, 30, 50, each with 30 independent runs, show that DPSO frequently outperforms standard PSO on multimodal problems, with improvements of 2-8× on functions such as Pinter, Ackley, and Levy, and up to 5× reduction in run-to-run variance. On unimodal landscapes the modulation term is counterproductive, confirming that DPSO targets the exploration-exploitation trade-off rather than offering a universal improvement. The method adds one hyperparameter, incurs 15–25% wall-clock overhead, and does not increase the asymptotic per-iteration complexity of PSO. The project code is available here: https://github.com/Kleyt0n/dpso

The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models
Seonglae Cho, Zekun Wu, Kleyton da Costa, Adriano Koshiyama.
[ArXiv 2026]

When a language model asserts that "the capital of Australia is Sydney," does it know this is wrong? Models assert misconceptions with the same fluency as facts, so the question cannot be answered from output uncertainty. Truth-related signals are known to exist in the residual stream, but not their geometry: how many dimensions carry the signal, how simple a detector can be, and whether it transfers. We characterize this geometry across 11 models (124M-14B) and test it causally with activation steering, concept erasure, and distributed alignment search. The structure is simple: two class centroids in a 2-8 dimensional subspace match a trained linear probe, and 25 labeled examples recover 90% of full-data AUC on GPT-2. Steering shifts hallucination rates by 9.1 points on six models, erasure drops detection to chance, and distributed alignment search, the only method that bounds rank, localizes at most five causal dimensions. The internal advantage is regime-specific: probes far outperform P(True) and semantic entropy on adversarial misconceptions but tie on standard QA. Single-dataset probes transfer near-randomly until joint multi-dataset training restores 0.73-0.91 AUC. That centroid distance matches probe performance indicates class separation is a mean shift, making detection geometric rather than learned.

Time series forecasting via integrating a filtering method: an application to electricity consumption
Felipe Leite Coelho da Silva, Josiane da Silva Cordeiro, Kleyton da Costa, Nemias Saboya, Paulo Canas Rodrigues, Javier Linkolk Lopez-Gonzales.
[Computational Statistics 2025]

Learning on Graphs via Generalized Divergence Measures
Kleyton da Costa.
[M.Sc. Thesis, PUC-Rio 2025]

This master dissertation investigates the effectiveness of generalized information measures for learning on graphs (LoG). The variational graph autoencoders framework proposed by Kipf and Welling (2016b) was modified by generalized divergence measures as part of the learning objective to delimit the research scope. Then, the main contributions of this work are: (i) the κ-divergences - a unified representation for generalized divergence measures; (ii) two novel families of divergences, δ and η; and (iii) the generalized graph variational autoencoders (GGVA) - a variational graph autoencoders framework based on κ-divergences. The experiments on LoG, using five citation network datasets and a Brazilian power grid network dataset, indicate that GGVA outperforms baseline models in node classification and link prediction, considering time efficiency and average precision. The qualitative analysis of the learned embeddings of GGVA indicates a good enough capacity to distinguish classes.

LibVulnWatch: A Deep Assessment Agent System and Leaderboard for Uncovering Hidden Vulnerabilities in Open-Source AI Libraries
Zekun Wu, Seonglae Cho, Umar Mohammed, Cristian Munoz, Kleyton da Costa, Xin Guan, Theo King, Ze Wang, Emre Kazim, Adriano Koshiyama.
[ACL 2025]

Open-source AI libraries are foundational to modern AI systems, yet they present significant, underexamined risks spanning security, licensing, maintenance, supply chain integrity, and regulatory compliance. We introduce LibVulnWatch, a system that leverages recent advances in large language models and agentic workflows to perform deep, evidence-based evaluations of these libraries. Built on a graph-based orchestration of specialized agents, the framework extracts, verifies, and quantifies risk using information from repositories, documentation, and vulnerability databases. LibVulnWatch produces reproducible, governance-aligned scores across five critical domains, publishing results to a public leaderboard for ongoing ecosystem monitoring. Applied to 20 widely used libraries, including ML frameworks, LLM inference engines, and agent orchestration tools, our approach covers up to 88% of OpenSSF Scorecard checks while surfacing up to 19 additional risks per library, such as critical RCE vulnerabilities, missing SBOMs, and regulatory gaps. By integrating advanced language technologies with the practical demands of software risk assessment, this work demonstrates a scalable, transparent mechanism for continuous supply chain evaluation and informed library selection.

Evaluating explainability for machine learning predictions using model-agnostic metrics
Cristian Munoz, Kleyton da Costa, Bernardo Modenesi, Adriano Koshiyama.
[ArXiv 2024] [PDF]

The rapid integration of artificial intelligence (AI) into various industries has introduced new challenges in governance and regulation, particularly regarding the understanding of complex AI systems. A critical demand from decision-makers is the ability to explain the results of machine learning models, which is essential for fostering trust and ensuring ethical AI practices. In this paper, we develop nine distinct model-agnostic metrics designed to quantify the extent to which model predictions can be explained. These metrics measure different aspects of model explainability, ranging from local importance, global importance, and surrogate predictions, allowing for a comprehensive evaluation of how models generate their outputs. Furthermore, by computing our metrics, we can rank models in terms of explainability criteria such as importance concentration and consistency, prediction fluctuation, and surrogate fidelity and stability, offering a valuable tool for selecting models based not only on accuracy but also on transparency. We demonstrate the practical utility of these metrics on classification and regression tasks, and integrate these metrics into an existing Python package for public use.

Deep learning model fragility and implications for financial stability and regulation
Rishabh Kumar, Adriano Koshiyama, Kleyton da Costa, Nigel Kingsman, Marvin Tewarrie, Emre Kazim, Arunita Roy, Philip Treleaven, Zac Lovell.
[Bank of England 2023]

Deep learning models are being utilised increasingly within finance. Given the models are opaque in nature and are now being deployed for internal and consumer facing decisions, there are increasing concerns around the trustworthiness of their results. We test the stability of predictions and explanations of different deep learning models, which differ between each other only via subtle changes to model settings, with each model trained over the same data. Our results show that the models produce similar predictions but different explanations, even when the differences in model architecture are due to arbitrary factors like random seeds. We compare this behaviour with traditional, interpretable, 'glass-box models', which show similar accuracies while maintaining stable explanations and predictions. Finally, we show a methodology based on network analysis to compare deep learning models. Our analysis has implications for the adoption and risk management of future deep learning models by regulated institutions.

Solid Waste Management in Peru’s Cities: A Clustering Approach for an Andean District
Katherine Quispe, Mayra Martínez, Kleyton da Costa, Hilario Romero Giron, José Francisco Via y Rada Vittes, Lizet Doriela Mantari Mincami, Mohamed Mehdi Hadi Mohamed, Alex Rubén Huamán De La Cruz, Javier Linkolk López-Gonzales.
[Applied Sci. 2023]

There is a great deficiency in the collection and disposal of solid waste, with a considerable amount disposed of in dumps instead of in landfills. In this sense, the objective of this research is to propose a solid waste mitigation plan through recovery in the District of Santa Rosa, Ayacucho. For this, a solid waste characterization plan was executed in eight days, and through ANOVA it was shown that there is a significant difference in means between business pairs except between a bakery and a hotel. Through clustering, zones A and B are highly correlated, reflecting that the amount of organic waste was greater than inorganic waste. In the organic waste valorization plan, the results through ANOVA indicate a significant difference for monthly and daily averages, and the clustering shows the different behavior of each month, drawing attention to August, concluding that the valorization pilot plan is viable due to the contribution of a large amount of organic solid waste to the valorization plant.

Anomaly Detection in Global Financial Markets with Graph Neural Networks and Nonextensive Entropy
Kleyton da Costa.
[ArXiv 2023]

Anomaly detection is a challenging task, particularly in systems with many variables. Anomalies are outliers that statistically differ from the analyzed data and can arise from rare events, malfunctions, or system misuse. This study investigated the ability to detect anomalies in global financial markets through Graph Neural Networks (GNN) considering an uncertainty scenario measured by a nonextensive entropy. The main findings show that the complex structure of highly correlated assets decreases in a crisis, and the number of anomalies is statistically different for nonextensive entropy parameters considering before, during, and after crisis.

Effects of Monetary Policy on Income Distribution: Evidence from Brazil
André de Melo Modenesi, Kleyton da Costa, Bernardo Modenesi, Norberto Montani, Sylvio Kappes, Fernando Nogueira da Costa.
[IE/UFRJ 2023]

This article investigates the impact of monetary policy on income distribution in Brazil. Income inequality affects both developed and underdeveloped economies, but its presence in the latter has a greater impact on vulnerable segments of society. The investigation of this phenomenon is critical for directing economic policies aimed at mitigating its adverse effects. We use macroeconomic variables and a Gini index calculated from microdata to measure income distribution. Our analysis employs vector autoregressive and Bayesian vector autoregressive approaches, regression analysis, and causality tests to find evidence of the impact of monetary policy on income distribution in the Brazilian case. The results show that a shock to SELIC and inflation positively impacts the Gini index, increasing inequality within a 95% confidence interval. However, an increase in economic activity and job generation has a negative impact on the Gini index, reducing income inequality observed in the economy.

Statistical and Artificial Neural Networks Models for Electricity Consumption Forecasting in the Brazilian Industrial Sector
Felipe Leite Coelho da Silva, Kleyton da Costa, Paulo Canas Rodrigues, Rodrigo Salas, Javier Linkolk López-Gonzales.
[Energies 2022]

Forecasting the industry’s electricity consumption is essential for energy planning in a given country or region. Thus, this study aims to apply time-series forecasting models (statistical approach and artificial neural network approach) to the industrial electricity consumption in the Brazilian system. For the statistical approach, the Holt–Winters, SARIMA, Dynamic Linear Model, and TBATS (Trigonometric Box–Cox transform, ARMA errors, Trend, and Seasonal components) models were considered. For the approach of artificial neural networks, the NNAR (neural network autoregression) and MLP (multilayer perceptron) models were considered. The results indicate that the MLP model was the one that obtained the best forecasting performance for the electricity consumption of the Brazilian industry under analysis.

Central bank digital currencies: a high-level overview
Kleyton da Costa, Antônio José Alves Júnior, Felipe Leite Coelho da Silva, Débora Mesquita Pimentel, André de Melo Modenesi.
[Preprint 2022]

In the aftermath of the private digital currencies’ revolution the debate on the costs/benefits of issuing a public digital currency gained prominence. This is a relevant public policy debate and, therefore, should not be led only by central bankers but also by academics. The aim of this paper is to discuss the properties and motivations of issuing a CBDC. A central bank-issued digital currency (CBDC) is a digital form of central bank money that differs from the current digital monetary forms made available by central banks – bank reserves and settlement accounts. When compared to other monetary forms (paper money, bank deposits, etc.) CBDC meets four key properties for a currency, which may be widely accessible, digital, central bank-issued, and token-based. Some countries have already launched guidelines and have been expanding their research and testing with digital currencies issued by their central banks, such as China (e-RMB), Brazil (Real Digital), and Sweden (e- krona). The conclusions indicate that the most relevant challenges for issuing a CBDC are: technical, operational limitations, legislative, and managerial barriers.

A machine learning approach to analyse ozone concentration in metropolitan area of Lima, Peru
Natalí Carbo-Bustinza, Marisol Belmonte, Vasti Jimenez, Paula Montalban, Magiory Rivera, Fredi Gutiérrez Martínez, Mohamed Mehdi Hadi Mohamed, Alex Rubén Huamán De La Cruz, Kleyton da Costa, Javier Linkolk López-Gonzales.
[Nature 2022]

The main objective of this study is to model the concentration of ozone in the winter season on air quality through machine learning algorithms, detecting its impact on population health. The study area involves four monitoring stations: Ate, San Borja, Santa Anita and Campo de Marte, all located in Metropolitan Lima during the years 2017, 2018 and 2019. Exploratory, correlational and predictive approaches are presented. The exploratory results showed that ATE is the station with the highest prevalence of ozone pollution. Likewise, in an hourly scale analysis, the pollution peaks were reported at 00:00 and 14:00. Finally, the machine learning models that showed the best predictive capacity for adjusting the ozone concentration were the linear regression and support vector machine.

Artificial Neural Networks and Conditional Heteroscedastic Models for Forecasting Exchange Rate
Kleyton da Costa, Felipe Leite Coelho da Silva.
[CNMAC 2021]

A Systematic Comparison of Forecasting for Gross Domestic Product in an Emergent Economy
Kleyton da Costa, Felipe Leite Coelho da Silva, Josiane da Silva Cordeiro Coelho, André de Melo Modenesi.
[ArXiv 2020]

Gross domestic product (GDP) is an important economic indicator that aggregates useful information to assist economic agents and policymakers in their decision-making process. In this context, GDP forecasting becomes a powerful decision optimization tool in several areas. In order to contribute in this direction, we investigated the efficiency of classical time series models, the state-space models, and the neural network models, applied to Brazilian gross domestic product. The models used were: a Seasonal Autoregressive Integrated Moving Average (SARIMA) and a Holt-Winters method, which are classical time series models; the dynamic linear model, a state-space model; and neural network autoregression and the multilayer perceptron, artificial neural network models. Based on statistical metrics of model comparison, the multilayer perceptron presented the best in-sample and out-sample forecasting performance for the analyzed period, also incorporating the growth rate structure significantly.